Project Ideas – Part 2 – Looking for contributors


Here are few more project ideas.

1. mobile/web app to record voice for wikisoure – Show a word, record it, upload to commons, link back to wiktionary.

2. mobile/web app to record audio books  – FreeTamilEbooks needs audio books too

3. wordpress to android app convertor – Why cant we convert a wordpress site as android app with RSS feeds?

4. epub to apk convertor – Let us publish ebooks as mobile apps too.

5. blog to epub convertor – fix, add images
https://github.com/sathia27/blog2ebook
Add a feature to download images and add them to ebooks.

6. Daily mobi files for tamil newspapers
Crawl newspapers daily, make mobi, send them to kindle in email daily.

7. Send to kindle – feature for FTE
Add Send to kindle feature to FreeTamilEbooks.com site

8. Lime survey – SAAS – alternate to google forms
Explore limesurvey and make it as alternate for google forms.

9. Collect politicians info and release as app, site

How can we collect all politicians details as education, assets etc and publish for public?

http://tshrinivasan.blogspot.in/2015/12/how-to-collect-details-of-TN-politicians.html

10. setup ELK for tamil literature search, build a search engine on top of it

Explore using ElasticSearch and Kibana for Tamil Text analysis.

11. fix android app to record audio for wiktionary –

https://github.com/Atul22/wikiAudio
done at https://meta.wikimedia.org/wiki/WikiConference_India_2016/Chandigarh_Hackathon

12. Analyse tamil tv/radio show audio, find how many english words are used/hour
This paper may help
https://www.ee.columbia.edu/~dpwe/papers/Wang03-shazam.pdf
https://mail.python.org/pipermail/chennaipy/2017-March/001429.html
Contact Ganesh for python implementation of this algo

13. gui for voice record/upload – wiktionary

https://github.com/tshrinivasan/voice-recorder-for-tawictionary

This needs a GUI version for windows users

14. gui for csv uploader

https://github.com/tshrinivasan/tools-for-wiki/tree/master/csv-uploader-wiktionary

This needs a GUI for windows users

15. gui for open-tamil font convertor

https://github.com/Ezhil-Language-Foundation/open-tamil

Need a web application or GUI for all features of open-tamil

16. mobile app to teach tamil – pollachi nasan

http://tshrinivasan.blogspot.in/2015/03/blog-post_9.html

17. wiki massuser create

Sometimes, we need to create 100s of users on wikipedia, for any training/event. Currently, only 6 users can be created. Admins can create multiple users, one by one. Automate this process using mechanize and beautifulsoup.

18. OCR4wikisource web version using google vision api

Rewrite https://github.com/tshrinivasan/OCR4wikisource with google vision api and give a web interface.

19. create a command line TTS from the source of a mobile TTS app.

Here is a open source TTS mobile app for tamil.

http://www.iitm.ac.in/donlab/tts/androidapp.php

Register and download the source and apk.
The voice named “Naveen” is good.

There are many c files in the folder
SSNFlitehtsTamil/app/src/main/jni

Can you compile those files and give a binary file as a command line tool?

Explore these code and share your thoughts on how to convert this as a
desktop/command line application so that we can use it in our
computers.

20. Create a GUI app for bulk photo uploader for http://commons.wikimedia.org

https://github.com/tshrinivasan/mediawiki-uploader

Project Ideas – Part 1 – Looking for contributors


contribute to open source க்கான பட முடிவு

I am listing here few project ideas and requirements. If you are interested in contributing to any open source project, consider these to start with.

I am giving an intro about each of them in this series of blog posts.

Add your comment here if you pick any of the project to do, so that others can join with you.

1. Clean up Epub files.

We create epub files for FreeTamilEbooks.com by using Calibre. It creates epub files with lot of extra span and other tags. We need to remove all the unwanted tags from those epub files.

Create a command line or web application to clean up the given epub files.

If you are writing in python, plan to create a calibre plugin to clean the epub files.

2.  Download reports for Tamil Wikisource Ebooks

http://ta.wikisource is providing ebooks downloads.

In this database, all language wiki source ebook downloads are stored.

http://tools.wmflabs.org/wsexport/logs.sqlite

Create a web application or command line application to get the details of tamil books and create a download
count report for each book.

Create similar report as http://freetamilebooks.com/htmlbooks/download-report.html

 

3. Improve FreeTamilEbooks android app

The android app for FreeTamilEbooks has some bugs.
https://github.com/jskcse4/FreeTamilEBooks/issues

Use the App and read the issues.
Fix them.

 

4. OCR4WikiSource – Create a web application

OCR4WikiSource is a command line application that connects google ocr and wikisource.
It sends the pdf files to google drive, ocr it, gets text, sends to wikisource.

Create a web application to upload any pdf file, send to google via google vision api, get text, send to wikisource.

Links:
Here is the requirement.
https://github.com/tshrinivasan/OCR4wikisource/issues/89

Few links about it.
https://goinggnu.wordpress.com/2015/12/28/announcing-ocr4wikisource/

https://goinggnu.wordpress.com/2015/09/30/automating-google-ocr-with-python/

https://meta.wikimedia.org/wiki/WikiConference_India_2016/Submissions/Introduction_to_OCR4WikiSource

Discussion with wikipedia developers on this.
https://phabricator.wikimedia.org/T120788

Google Vision API
https://cloud.google.com/vision

Explore the links

https://github.com/GoogleCloudPlatform/cloud-vision

http://terrenceryan.com/blog/index.php/working-with-cloud-vision-api-from-php/

https://github.com/thangman22/google-cloud-vision-php

http://blog.aimanbaharum.com/2016/04/21/ocr-with-google-cloud-vision-api/

 

5. FlipBoard like application for Tamil

Flipboard is a web, mobile app which gives latest content on user selected topics. Create such application for providing tamil content from web on various topics. Content contributors should give links for good articles with relevant categories, tags. Users should subscribe to categories and read the latest content.

 

6. Firefox plugin for tamil wikisource proofreading

 

Tamil wikisource is having around 2000 public domain ebooks, OCRed by google OCR. We have to proofread those books manually.
QuickWikiEditor is a Firefox plugin that enables on the page editing of wiki content.
https://addons.mozilla.org/en-US/firefox/addon/quickwikieditor/

Need to extend this plugin, to send the error words and the corrected words to a remote web application. From there, we can get the list of error words, search for them in entire ta.wikisource.org, replace with the corrected words automatically using bots.

Extend the plugin and create a web application to get the words collection from the plugin.

 

7. Fix the Tamil TTS by IITM

IIT Madras and SSN college, released a Text to speech application for Tamil, as an android application. You can get the source at
https://www.iitm.ac.in/donlab/tts/

It is very initial version. Not as good as the latest  web version available at http://speech.ssn.edu.in/

 

Still, we can learn, extend the initial version.

Explore the android app, get the C code out of it, create a command line app or web app as having the c code as backend.

 

8. Web application to add details about ebooks in a xml file, in github.
We release Tamil ebooks at FreeTamilEbooks.com

We store all the details about the books in a XML file.

This file is source for Android and iOS apps for FreeTamilebooks.

Once an ebook is released, we have to update the xml file manually, which is tough for non-tech contributors.

Need a web application to get the ebooks details in a form, then add those details in XML file and commits to the repo automatically.

 

9. Add ebooks automatically in GoodReads.com

We can add the details about the ebooks in FreeTamilebooks.com to GoodReads.com

We have to fill a long form manually.
Need a command line or web application to simplify this process or automate it for adding info about the books in FreeTamilEbooks.com

10. Build a SAAS version of planet kind of RSS aggregation software.

 

Most tech communities need a planet kind of RSS aggregation software. They have to buy a VPS, install planet software and add the RSS feeds.

It will be good, if we build a SAAS version of planet or similar software, so that they can simply sign in, add rss feeds and start using it.

There are more ideas. Written them somewhere on my notebooks. Will collect them and share soon.

All the projects should be released as Free/Open Source software only.

If you are interested in doing any of the things said above, comment here.

email me to tshrinivasan AT gmail DOT com to know more details on any of the project.

unrtf – rtf to html conversion utility


For http://FreeTamilEbooks.com project, we have to convert many Word Documents into html before converting them to epub using http://PressBooks.com

I used LibreOffice to open Word doc and to save as HTML.

Till LibreOffice 4.1, the images are extracted and stored separately along with HTML file.

But, after LibreOffice 4.2, they moved to base64 type of encoding of images, so that images are embedded into HTML files. We can not separate images from HTML files.

This was so annoying and many people are reporting this as a bug here.
https://bugs.documentfoundation.org/show_bug.cgi?id=48887

But, This seems not to be fixed.

So, I installed LibreOffice 4.1 in /opt just to use the old feature of storing images separately.

Just now, found another utility, unrtf to do the same.

http://www.gnu.org/software/unrtf/

To install it in ubuntu/debian

sudo apt-get install unrtf

If you get an word doc with images, save it as a Rich Text File using libreoffice writer.

example:

test.doc -> test.rtf

Then,

unrtf test.rtf > test.html

This gives a nice HTML file and images separately.

Thanks for the GNU team for the nice utility.

I can get rid of old LibreOffice 4.1 now.

How to create an ebook for Free Tamil Ebooks.com?


FreeTamilEbooks.com is a portal for Tamil ebooks with creative commons license.

Let us see here, how to create ebooks for this portal.

1. Find content with creative commons license.

There are many blogs/sites in Tamil with creative commons license. We can get the content from there and create ebooks. We shold mention the same license in the books.

Ask bloggers and writers to release their content in creative commons license, so that we can create ebooks with their works.

2. Create an account in Pressbooks.com

We use Pressbooks.com to create the ebooks. It can export in the epub, mobi and XHTML formats.

Create login in pressbooks.com

create a ebook.

There are front matter, main body and back matter sections.

Add new chapters in these sections.

Front matter.

1. Intro page

Add the cover image, intro text, author name etc. Cover image should be in vertical size. Check freetamilebooks.com for sample cover images.

2. License Page

3. Ebook created by and publisher information

Main Body

1. Chapter 1

2. Chapter 2

3. etc

Copy and paste the book’s content here as new chapters.

Back Matter

1. About the author

2. Any other notes from author

3. About Free Tamil Ebooks.com [ copy the content from http://freetamilebooks.com/about-the-project/ ]

4. Add custom css for ebook.

Goto Apperance – Themes- Activate PressBooks custom css

Click the custom css. Edit CSS

Choose “Ebook” for the option “You are currently editing CSS for: “

Then, paste the following content and save.

p {
display : block;
}

5. Book Info

Add the following details in the book info.

Title, Author, Publisher, Publication Date, Language(Tamil), Add cover Image.

Copyright year, Copyright holder (author name), copyright notice ( add relevant creative commons license text here)

6. Export

Goto Export page.

Export as epub, mobi and xhtml.

Check the epub in (http://readium.org ) chrome plugin.

You can stop here and send the epub, mobi, XHTML files to freetamilebooksteam@gmail.com

We will take care of PDF files creations.

If you wish to produce A4 PDF and 6 Inch PDF, follow below instructions.

7. Creating PDF files:

Open the XHTML file in a text editor.

Add the following CSS code inside the <head> </head> block of the file.

We keep updating the CSS for improving the readability of the PDF files.

Get the latest CSS code from

https://gist.github.com/tshrinivasan/95ace5882cac6a1070fe

<style type=”text/css”>

h1.front-matter-title{
page-break-before: always;
}

#toc {
page-break-before: always;
}

div.chapter-title-wrap{
page-break-before: always;
border-bottom-style: groove;
display:block;
margin-bottom:100px;
color : #3366FF;
}

h3.chapter-number {
display:none;
}

h3.front-matter-number {
display:none;
}

#half-title-page {
display:none;
}

@media print {
img {
max-width: 100%;
height:100%;
}
}

</style>

In the TOC section, replace the word “Contents” with பொருளடக்கம்

Remove the line “<li><a href=”#main-body”>Main Body</a></li>”

Save the file.

8. Firefox Hacks:

We need to print A4 PDF and 9cm x 12 cm sized 6 Inch PDF file from Firefox browser.

In GNU/Linux, we can print as PDF file from firefox.

In windows, install a free PDF printer as doPDF.

Remove margins in Firefox

Firefox has its own margins for printing. We have to remove them all.

Type about:config in the addressbar.

Search for the string “margin”

printer_Print_to_File.print_margin_bottom
printer_Print_to_File.print_margin_left
printer_Print_to_File.print_margin_top
printer_Print_to_File.print_margin_right
printer_Print_to_File.print_unwriteable_margin_left
printer_Print_to_File.print_unwriteable_margin_bottom
printer_Print_to_File.print_unwriteable_margin_right
printer_Print_to_File.print_unwriteable_margin_top

Set the value as 0 for all the above strings.

Now, open the html file in a new tab.

File->Print

Select Print to File

In Page setup tab, select paper size as “Manage Custom Sizes

Add new paper size named “kindle”

width: 90 mm height 120 mm

margins: top : 10mm bottom : 15mm left: 5 mm right 5mm

save this.

Add new paper size named “Custom A4”

width: 210 mm height: 297 mm

margins: top:25mm bottom:35mm left:25mm right 25mm

save this.

These are one time tasks only. We can use the same settings in our future works.

Now, print as two PDF files.

One with paper size “kindle” and 85% scale.

Another with paper size “Custom A4” and 100% scale.

Thats all. We got the epub, mobi, A4 pdf and 6 Inch PDF files for the ebook.

Now send these 4 files and the cover image to freetamilebooksteam AT gmail DOT com.