Wednesday, October 22, 2014

A Social critic, His "Ignorance" and an adage,"Absence of Evidence is not Evidence of Absence"

A noted Social critic on Facebook

"മലയാളി എന്നെങ്കിലും ദീപാവലിയെ ഒരു ആഘോഷമായി കൊണ്ടാടിയിട്ടുണ്ടോ എന്നെനിക്കറിയില്ല.അതിനാല്‍ ദീപാവലി ആശംസിക്കാന്‍ സത്യസന്ധമായി പറഞ്ഞാല്‍ എനിക്ക് കഴിയുന്നില്ല.എന്‍റെ ഓര്‍മയുടെ സംസ്കാരത്തില്‍ അത് ഒട്ടും ഫിറ്റ്‌ ആകുന്നില്ല.ആശംസിക്കാത്തത്തിനു പിണങ്ങരുത്.ചില കുറവുകളും മറ്റു ചില അധികങ്ങളും ഉള്ള ജീവിതമാണ് നമ്മുടേത്‌.

ആദിവാസികള്‍ ഒരു തുണ്ട് ഭൂമിക്കായി നടത്തുന്ന "നില്‍പ്പ്സമരം" ഈ ദീപാവലിയുടെ ഒരു സവിശേഷതയാണ്.
അവരെ വംശഹത്യ ചെയ്യാന്‍ മുന്നണിയായി പ്രവര്‍ത്തിക്കുന്ന മത രാഷ്ട്രീയ ദുര്‍ഭൂതങ്ങള്‍ക്കു ഒരു പക്ഷെ നരകാസുരന്റെ മുഖച്ഛായ ഉണ്ടാകാം.സംഹരിക്കാന്‍ ഒരു അവതാരം ഉണ്ടാകുമോ..?അതോ നമ്മള്‍ കൂട്ടായി അത് ചെയ്യണോ..?
ദീപം തെളിഞ്ഞിരുന്നെങ്കില്‍......!
ദിവസം 108 ആയി അവരുടെ സമരം....


"

In Kerala, Trivandrum and Kollam district celebrate Diwali with great fan fare. Where as northern districts celebrate Vishu. 

In Trivandrum, Vishu is not a great celebration. In the year 1999, I did not see any kids who come to the door for  "Vishu Kaineetam". Curious, I asked a relative of mine and he told me that Vishu is mainly celebrated in Alleppy and the above part of Kerala. (Of course, Vishu is celebrated as "Baisaki" all across India as well)

Our universe is very vast and despite being a enlightened person, the critic in question could not understand some basic facts about the milieu he lives.


I think, in the above musing, there is mixing of two issues. The issue of Tribal land is a genuine one and why it should be mixed with Diwali? Creating awareness on Social issues, by making a statement which is not fully true, is suitable for "Demagogues". If usage of Diwali is used to take the metaphors for his explanation, that is fair enough. But, I do have  a difference of opinion only in the assertion.

I think, first and the foremost, the Peron in question suffers from, "If I have not see in it, It does not exist". Are people of Trivandrum and Kollam not Malayalis?

My family is into the business of selling Crackers and we do good business during this time. From 1977 on-wards (I started going to our shop from that year), I know from my first hand experience, that lot of Malayalis do celebrate Deepavali. 

If it is not celebrated, how a word will enter Malayalam dictionary. I think, in Nair/Ezhava calendar, it might not be a big celebration. 







Tuesday, October 21, 2014

Going for the IOT Kochi meetup today (22nd October) @ Startup Village, Kalamassery.

From my Facebook wall, I learned that the first meeting of the Kochi IoT meetup will be held today evening around 6.00 pm (22nd October,2014) @ Startup Village, Kalamassery. I have promptly registered for the event.

You can register for the event here. The agenda is given below for your reference. 

Inauguration of Kochi IoT Meetup and first session.
                
                                         





Venue : 3rd Floor, Startup Village, Kalamassery
AGENDA

• Registration  and Dropping business cards
• Inauguration of IoT Kochi Meetup by moderator (15 mins)
• Technology landscape of IoT (45 mins)
• Refreshments and socialization 
• Lucky Dip !!

Monday, October 20, 2014

Concurrency vs Parallelism - what exactly is "brewing"?

These twin terms are used as synonyms whenever people talk about Multi-Core programming, Threading, Synchronization, Tasks and most things to do with modern programming.

These two are not exactly synonyms, where as people use ( I have as well) these stuff interchangeably. At a closer inspection, at a subtle level, Concurrency is a problem which we have to deal with and Parallelism is a solution to certain problems which is amenable to simultaneous execution of sub-tasks to finish a large task.

According to Rob Pike,
          Concurrency is all about dealing with lot of things at Once (Reactive/Event)
          Parallelism is about doing lot of things at Once (Action)

Traditional programs are Sequential in Nature and they execute series of tasks in a sequential manner. Advent of multi-threading did add complication there, as large number of simultaneous threads execute things in sequential way (on a per thread basis)  and this can lead to problems when accessing shared mutable state.

Shared state is not a problem, If threads are not mutating the state. If things are mutable, we need to put synchronization primitives around it. Relational database access is a potential source for Concurrency issues. By setting the proper isolation levels and desgining the system in a partitioned manner, we can limit the concurrency issues. Sometimes, one will do a use case analysis to focus on critical paths for concurrency issues and leave the rest for luck.

In a digital computer, Parallelism can be @

  • Bit Level Parallelism - we can use a 32 bit or 64 bit CPU for doing computation. More width the data, address and control bus have, we will be able to process things faster
  •  Instruction Level Parallelism (ILP)- Modern CPUs can execute multiple floating point and integer operations simultaneously. To exploit that, the compiler has got generate code by doing complex dependency analysis. 
  • Data Parallelism - Image processing (especially point processing),  certain Geometric computations can be done simultaneously as they are data parallel operations
  • Task Parallelism - We can exploit multiple processors to achieve task parallelism. Ray tracing is a graphics operations which can exploit this parallelism
  • Pipeline Parallelism - Certain computations can be divided into stages where each stage will execute and exploid data/task parallelism available. Things like pipeline stall, bottleneck are problems to watch for in the case of Pipeline architecture
Next time, when you hear Concurrency and Parallelism, you have got some conceptual tools to distinguish both.




An "AweSome" (OSEMN) look at Data Science

According to a popular definition of Data Science, it is the art, craft and science of

  • Obtaining Data
  • Scrubbing Data
  • Exploring Data
  • Modelling Data
  • iNterpreting Data
It is called OSEMN model and it is pronounced as Awesome. Let us looking into what each of the steps do, in a nutshell

  • Obtaining Data - Generate, Query,Extract, Download data from various sources. Sources can be Survys/Data generation tools/Sensors (Generate), Databases and Social media sites (Query),Documents/Excel Files(Extract) and Web pages(download) etc.Storage and retrieval for future activities are also handled here. 
  • Scrubbing Data- Filtering, Cleansing, Missing value detection, Outlier detection,Format conversions,Normalization,Cannonicalization etc. fall in here. 
  • Exploring data - Inspection for patterns, Derive summary statistics, Visualization etc. falls under this category
  • Modelling Data-Techniques like Inferential statistics, Classification, Clustering, Regression, Ensemble models, Dmensionality reduction etc. come here.
  • iNterpreting Data - Generate Conclusion, Evaluate the consequence of the results, Summary report and Visualization come under here.
The last two steps are mostly done by Data Scientist proper (Modellers) and Domain Experts, who have got intimate familiarity with the data. 

It is not a linear process and lot of trial and error is involved in the whole game.

As a Software developer where can I place my bet?

  • Obtaining Data
  • Scrubbing Data
  • Exploring Data ( with Domain expert)
  • Writing R/Python scripts for realizing Models as scripts ( executable artifacts)
  • Producing Visualziation in Software based on the request of Domain exprt/Modeller

If you take a closer look, these points are the primary concerns of Data Engineering. 

Residential Entrance Coaching - A Malappuram sexual harassment episode

I Just happen to read in Mangalam daily, about the arrest of one Mr. Jalaludin Ahmed for raping girls and performing homo sexual acts on boys. The arrest happened at Vengara, Kannamangalam from a "private institution"(?) run in the Jamat-e-Islami run school.

The news clip can be retrieved from here. I am pasting it out here.

മലപ്പുറം: എന്‍ട്രന്‍സ്‌ പരിശീലന കേന്ദ്രത്തിലെ ഹോസ്‌റ്റലില്‍ കഴിയുന്ന 30 ഓളം വിദ്യാര്‍ഥിനികളെ വിവിധ തരത്തില്‍ പീഡനം നടത്തിയ അധ്യാപകന്‍ അറസ്‌റ്റില്‍. വേങ്ങര കണ്ണമംഗലം ചെറേക്കാട്ടെ ഐഡിയല്‍ ക്യാമ്പസിലെ സ്വകാര്യ സയന്‍സ്‌ ഇന്‍സ്‌റ്റിറ്റ്യൂട്ടിലാണു പീഡനം നടന്നത്‌, പെണ്‍കുട്ടികളെ ലൈംഗികമായും ആണ്‍കുട്ടികളെ മര്‍ദ്ദിച്ചും പീഡിപ്പിച്ചകേസിലാണു നടത്തിപ്പുകാരനായ അധ്യാപകനെ പോലിസ്‌ അറസ്‌റ്റ് ചെയ്‌തത്‌. എ.ആര്‍ നഗര്‍ സ്വദേശി ഏറിയാടന്‍ ജലാലുദ്ദീന്‍ അഹമ്മദാണു(32)അറസ്‌റ്റിലായത്‌.

ജമാഅത്തെ ഇസ്ലാമിയുടെ നേതൃത്വത്തില്‍ ചെറെക്കാട്‌ ധാര്‍മ നഗരിയില്‍ നടന്നുവരുന്ന ഐഡിയല്‍ സ്‌കൂള്‍ ക്യാമ്പസില്‍ താല്‍കാലികമായി സ്വകാര്യ വ്യക്‌തിക്ക്‌ സ്‌ഥാപനം നടത്താന്‍ അനുമതി നല്‍കിയിരുന്നു. ഇവിടെ പ്രതി 'മാര്‍ക്ക്‌' എന്ന പേരില്‍ ആരംഭിച്ച സയന്‍സ്‌ ഇന്‍സ്‌റ്റിറ്റ്യൂട്ടില്‍ പീഢനം നടന്നതായി കഴിഞ്ഞ ദിവസം ചൈല്‍ഡ്‌ ലൈനില്‍ പരാതിലഭിച്ചിരുന്നു, ഇതേ തുടര്‍ന്ന്‌ നടത്തിയ അന്വേഷണത്തിലാണ്‌ പോലീസ്‌ കേസെടുത്ത്‌ പ്രതിയെ അറസ്‌റ്റ് ചെയ്‌തത്‌. സ്‌ഥാപനത്തില്‍ അന്യജില്ലക്കാര്‍ ഉള്‍പ്പെടെ 62 വിദ്യാര്‍ഥികള്‍ പെണ്‍കുട്ടികള്‍ ഉള്‍പ്പെടെ താമസിച്ച്‌ പഠനം നടത്തിയിരുന്നു
.
രാത്രികാലങ്ങളില്‍ പെണ്‍കുട്ടികളെ മന:ശാസ്‌ത്ര ക്ലാസിന്റെ പേരില്‍ ക്യാമ്പസിലേക്കു വിളിച്ചു വരുത്തി ലൈംഗിക ചൂഷണം നടത്തുന്നതായും ആണ്‍കുട്ടികളെ പ്രാകൃത രീതിയില്‍ പീഡിപ്പിക്കുന്നതായുമാണ്‌ പരാതി. ചൈല്‍ഡ്‌ വെല്‍ഫയര്‍ കമ്മിറ്റി ചെയര്‍മാന്‍ അഡ്വ. ഹാരിസ്‌ പഞ്ചിളി, ചൈല്‍ഡ്‌ ലൈന്‍ പ്രവര്‍ത്തകരായ അന്‍വര്‍ കാരാക്കാടന്‍, മുഹ്‌സിന്‍ പരി, സീതാദേവി, നാവാസ്‌ കൂര്യാട്‌ എന്നിവരുടെ നേതൃത്വത്തിലാണ്‌ തെളിവെടുപ്പ്‌ നടത്തിയത്‌. പ്രതിയുടെ വസ്‌തിയില്‍ നിന്നാണു പോലീസ്‌ അറസ്‌റ്റ് ചെയ്‌തത്‌. എസ്‌.ഐ: സി.എ ബാല ഗംഗാധരന്‍, സിവില്‍ പോലീസ്‌ ഉദ്യോഗസ്‌ഥരായ ഷൈജു, ദീപ എന്നിവരാണ്‌ കേസന്വേഷിച്ചത്‌. മലപ്പുറം കോടതിയില്‍ ഹാജരാക്കിയ പ്രതിയെ റിമാന്‍ഡ്‌ ചെയ്‌തു.

Jalaludin was running an entrance coaching center in the campus of the Jamat-e-Islami run school. By the pretext of giving psychological counselling, he is alleged to have harassed girls and even did not spare boys.

It is ironical that, Mainstream media has not reported this or nor a picture of this man shown anywhere. May be, it is a "private matter" for them!. Like a friend asked long back,"What Is your problem, If a Muslim is having sex with other Muslims"?





Data Problems take time to Solve, Programmers should not waste time on it

In the year 2012, a person got in touch with me to write some indicators for MetaTrader platform. I was new to the platform, but, had some idea about Technical Analysis. I could understand what the platform does and it's execution model within minutes.

After a week of fiddling with the software , I was comfortable in writing scripts which can detect some  patterns. I had  three partially written scripts by an American company as well. Armed with these stuff, I set out to write the indicators my client wants.

Essentially speaking, I was trying to fit a line to a curve. I was successful in some places and when I tan the algorithm on past data, it shows unpredictable behavior. I tried for a month and could not rectify the issue.

The problem was not with my algorithm. The primary reason was there are no underlying patterns everywhere. Or in mathematical terms, there is no continuity. Even-though I started out to fit some kind of line or curve to a data points,I was trying to fit data points to my line.

I understood one thing, a Junior programmer with a domain expert has to struggle for months to have some chance for success. I called it a "Data Problem", which means, it is not an Algorithmic issue.

Behind Data Science, lurks a lot of  Data problem. Ask a domain expert to solve it for you. As a programmer you should focus on Automating the solution.



Assembly output of a C++ Template Metaprogram

















To demonstrate that C++ template mechanism expand templates during the compilation of the source code, I wrote a small C++ program.

///////////////////////////////////////
//  BIT_SET.cpp
//  A C/C++ code snippet to demonstrate
//  the computation during the compile time
//
//  In GCC
//  ------
//    g++ -S BIT_SET.cpp 
//    The above command generates Assembly
//    output (BIT_SET.s)
//
//  For generation of executables
//    g++ -o temp.exe BIT_SET.cpp
//
#include <stdio.h>

/////////////////////////////////////
// A Simple template class to count
// the number of bits set in a byte
//
// BIT_SET<4>::RESULT will give 1
// BIT_SET<12>::RESULT will give 2
// BIT_SET<127>::RESULT will give 7
// 
// All of the computation will be done by 
// the Template code generation mechansim
// during the compile time
//
//
//
template< unsigned char byte > 
class BITS_SET
{
public:
    enum {
     B0 = (byte & 0x01) ? 1:0,
     B1 = (byte & 0x02) ? 1:0,
     B2 = (byte & 0x04) ? 1:0,
     B3 = (byte & 0x08) ? 1:0,
     B4 = (byte & 0x10) ? 1:0,
     B5 = (byte & 0x20) ? 1:0,
     B6 = (byte & 0x40) ? 1:0,
     B7 = (byte & 0x80) ? 1:0
    };

public:

    enum {
        RESULT = B0+B1+B2+B3+B4+B5+B6+B7
    };


};

//////////////////////////////////
//
// The entry point
//
//
//
//
int main( int argc, char **argv)
{
   int a = BITS_SET<127>::RESULT;
   int b = BITS_SET<116>::RESULT;

   printf("%d\n",a+b);

}

If you take a look at the image, you can see a will be assigned 7 and b will be assigned 4 in the assembly language generated. The answer you get is eleven. I am attaching the console output below



U26646@C666LTRV ~/cpp
$ g++ -o test.exe BIT_SET.cpp

U26646@C666LTRV ~/cpp
$ ./test.exe
11

U26646@C666LTRV ~/cpp
$ g++ -s BIT_SET.cpp

U26646@C666LTRV ~/cpp
$ cat BIT_SET.s
        .file   "BIT_SET.cpp"
        .def    ___main;        .scl    2;      .type   32;     .endef
        .section .rdata,"dr"
LC0:
        .ascii "%d\12\0"
        .text
        .globl  _main
        .def    _main;  .scl    2;      .type   32;     .endef
_main:
LFB7:
        .cfi_startproc
        pushl   %ebp
        .cfi_def_cfa_offset 8
        .cfi_offset 5, -8
        movl    %esp, %ebp
        .cfi_def_cfa_register 5
        andl    $-16, %esp
        subl    $32, %esp
        call    ___main
        movl    $7, 28(%esp)
        movl    $4, 24(%esp)
        movl    24(%esp), %eax
        movl    28(%esp), %edx
        addl    %edx, %eax
        movl    %eax, 4(%esp)
        movl    $LC0, (%esp)
        call    _printf
        movl    $0, %eax
        leave
        .cfi_restore 5
        .cfi_def_cfa 4, 4
        ret
        .cfi_endproc
LFE7:
        .ident  "GCC: (GNU) 4.8.3"
        .def    _printf;        .scl    2;      .type   32;     .endef

U26646@C666LTRV ~/cpp
$

}


Sunday, October 19, 2014

Some links on Templates and Meta-programming in C++ (from my blog)

I had taken a small presentation on C++ Template meta-programming at the Ruby meetup held yesterday. I had blogged about my presentation here. A friend of mine asked for some pointers regarding the stuff I have blogged.  

The Idea of "Zero Cost Abstraction in terms of execution speed is dealt here.



Abstraction cost in terms of intellectual complexity lead to the demise
of  C++ in certain quarters.





Compile time Computations in C++. Shortest
Template metaprogramming program which does
all the computation at the compile time





Difference between Meta Object Protocol (runtime) and Template Metaprogramming




Erwin Unruh and the birth of C++ Tempalte meta programming is available @




The following C++ program demonstrates how you can create a SmartInt class with all the expressiveness an Integral type can get ( Int, long) etc. This shows expressiveness support of C++. A user defined class can be as close to the built-in type.

By using the Smartint class, there are examples available where STL sort algorithm and for_each is being demonstrated. A look at C++ lambda is also available there. This shows the substiutability of user defined types where  a built-in type is expected.

Of course, to achieve this, the programmer has to know quite  a lot!. This demonstrates that "Zero Cost Abstraction" is only in execution speed. Not, in terms of Cognitive load.




Lack of Seamless abstraction at the language level is another cause of the  "death of C++". An enhanced version of this article is published by Technopark Today is also available.



Some thoughts on Advanced C++ is available here @



Advocating the usage of Advanced features of C++ in a minimalistic manner is advocated here. Some notes on Templates and MetaProgramming are availble here.



A review of the book,"Modern C++ Design", which uses most of the Metaprogramming constructs of C++ is available here @



An essay about cognitive blindness


Some highlights of my presentation on C++ Template Meta programming @ Ruby meetup on October 19th



Yesterday, I had gone to attend Ruby meetup and blogged about it here. Ruby is a language which has got run time meta programming as a built-in feature.Basically, It means Ruby allows you to play with it's Abstract Syntax Tree (AST) during the run time. It is a "civilized" form of Self Modifying code.

I took a presentation on "C++ Template Meta Programming" to explain the compile time meta programming features of C++. Of course, C++ templates was not designed originally to do that.

Here are some of the key take aways

  • C++ and it's promise to give the User defined Types(Classes) the same expressive power of Built-in types
  • Potential for substitution of our types on standard data types to customize Generic Algorithms. 
  • Zero Cost Abstraction - You do not pay the price in terms of Execution speed
  • Abstraction leak in the case of C++  
  • What are templates and how it is different from Generics in C#/Java
  • Compile time expansion of Templates, including the Recursive templates
  • While Expanding recursive templates, C++ act as an interpreter
  • Erwin Unruh and his serendipitous discovery of Turing completeness of C++ template mechanism. 
  • Reflection API of Java/C# ( aka Procedural reflection)
  • Meta Programming == Declarative Reflection
  • Some Applications
We had some interesting discussion about Concurrency vs Parallelism, Go language and it's concurrency support (my criticism here as well) etc.




Ruby meet up on 19th October - A personal summary

Yesterday, I started around 1.40 pm to attend the Ruby meetup being held @ Palarivatton,Eranakulam. I had some difficulty in spotting the location and finally managed  to reach the venue around 2.25 pm. The meeting was supposed to start around 2.00 pm and only five people were present ( Later it became nine, If you count my son who browsing the net in the same hall)

I saw  Nithin, Deepak Kumar and Revath Kumar etc. there (They are people whom I know really well).

Nithin Bekal (the person who is supposed to present on Meta-programming) was wearing a Banian of an event held at Bangalore. The event, Functional Conference was held between October 9 and 11. The event had the presence of Bruce Tate,Debasis Ghosh (DSLs in Action fame) and Nithin gave in a nutshell Bruce tate's talk surrounding around the "Fear and evolution of programming languages", in the context Concurrency,Many core Parallelism and Functional Programming.


Officially, there was only one session and Nithin started around 2.45 pm. Some of the highlights of his excellent presentation were

  • Memoization ( aka Caching) using Meta Programming techniques of Ruby
  • Injection of instance method and static method 
  • Ubounded methods and binding 
  • Tail Call Optimization - Python vs Ruby philosophy
  • Tail Call Optimization, How to implement in Ruby using Dynamic Compilation
  • How to simulate Java like keywords in Ruby 
  • The previous example convinced me why Ruby is great for writing DSL
This is one of the most advanced presentation I have seen in Kerala. 

Harishankar took a session explained about the predictive analytics done by Facebook to detect whether your password is hacked. He then went to the Netflix Open Source portal ( I did not knew it existed) and explained about the Scumblr project. This session kick-started a wonderful discussion on Scrapping. I came to know that libxml can create DOM tree really well from the arbitrary HTML page, even If it is not well formed (from Deepak Kumar) . Finally, I can write better scrapers to do some specific tasks.

I gave a presentation on C++ Template meta programming, to explain the difference between  Compile time meta programming and Run time meta-programming of the Ruby language.

In the end, we discussed lot of stuff about Computer programming, Functional Programming, Electronics, Digital Engineering, Top down learning vs Bottom up learning, Malayalam movies, AT Kovoor's book.

PS:- When the discussion reached MT Vasudevan Nair's "Sukrutham", a film based on the story written by MT Vasudevan Nair, I came to know about one fact. MT Vasudevan Nair was working for Mathrubhumi and he fell ill for  a long time. When he returned to his seat, he saw an obituary and news clip to publish regarding his death. The information was given by Sarun, the person who is a promoter of the firm which gave us the venue.












Saturday, October 18, 2014

Data Engineering vs Data Science - a software developer can become the former,not the latter

Now a days, people are fascinated with Data Science. The umberla term for all activities related to managing humungous amount of systems and people generated data.

What exactly is Data Science?

Scientists world wide, after Francis Bacon became empiricist. They conducted experiments to collect data to prove or disprove conjectures, hypothesis and laws. This is the wonderful world of Small data. 

Now, that we have got plenty of data, why cannot we generate actionable insights based on statistical and machine learning techniques?. In this way, we do not need to conduct calibrated experiments. Since we have have got plenty of data, a statistical approach will give reasonably good approximate data. 

What is data Engineering?

Data Engineering is the process of extracting, aggregating, transforming, cleansing, loading data into some kind of structured format amenable for Data Scientist to perform Analysis. It also includes implementation of Applications which realizes Machine learning models as executable software artifacts.

Data Engineering uses Search Engine libraries, Machine Learning libraries, Statistical packages etc. to write applications to realize Learning models like Recommendation systems,Clustering, Classification, Regression Analysis, Ensemble Algorithms, Apriori Algorithm etc.

The systems like Hadoop, Distributed Algorithms, Pig, Hive, Cascading are tools for Data Enginers.


Why Data Engineering for Programmers?

Data Engineering can be treated as a branch of Software engineering. The skill set in Software Engineering can be easily transferred to the realm of Data Engineering. Data Engineers need to only learn Data Science from a functional/transformation perspective. Data Engineer's concerns are What are the input vectors?, understanding of the general computation underpinnings of algorithms/model, key parameters which can affect the output vector and qualitative interpretation of quantitative data in the domain of problem analysis etc.




Do not bother about Grammar, Write what you have!

Kurt Godel is an Austrian Logician who came up with Indeterminacy principle, in the case of Mathematics. His startling conclusion was,

"Mathematics can be either consistent or complete. It cannot be consistent and complete at the same time"

A wild (or even flawed) extrapolation of  the stuff can be done about Writing

"You can either be Grammatically consistent or complete. Not both. If you look for Grammatical correctness, you cannot communicate effectively. To communicate effectively, you have to be bit lenient on grammar. Then, you are not complete"

My twelve year old son has started blogging about things he like. My wife relented to his repeated pestering to create a blog site for him. So far he has got nine blog posts. It is available here.

Other day, my wife, after reading his blog told my son that there are lot of grammar mistakes. I had even complaint that he uses capital letters and it is equivalent to shouting.

Suddenly, I realize that, let him write it. Once it picks up a volume, slowly grammar bug will get induced into him. If we try to be correct from the word go, you won't express anything.  I improved my Grammar habit based on the feedback given by my friends, after reading my blog posts.




Are you suffering from "Pre-Web Syndrome"?

Recently, two friends of mine had a discussion with each other and the conversation drifted to the way I monetize (relative) the knowledge acquired. One of the guy in the duo who were engaged in the discussion is an Intellectual type, whose cognitive apparatus is one of the best a human can inherit. Let us call him Gopi

Gopi has got Pre-Web syndrome. "Pre-Web Syndrome" is a term coined by me to describe phenomena where people are stuck up in the way humans conducted social affairs, trade and cultural matters etc, before the advent of the Web. The year 1989 can be considered as the year of the birth of the World wide web. ( Only by 1995, it came to India though. You have to dial to US to access the internet in those days)

Gopi made a remark which referred me ( I cam to know about this because the other friend accused me of the same after getting influenced by Gopi!)

"When I study something, I really go about it in full. Only after getting deep understanding, I will talk about the subject matter. Whereas, Praseed will speak about something moment he has got cursory understanding of the matter. . What will happen, If something turns out to be wrong? He is able to get away with things because lot of fools around him think that he is an expert in what he says and they are deluded into it because of his reputation as a computer programmer"

I got pissed off with this. But, after contemplation, I came to know about lot of things. 
  • Gopi is a  perfectionist. Ability to live in an imperfect and fuzzy world matters at the social realm. 
  • Gopi do not have idea about how the human cognitive mechanisms work. He thinks that life is a Mathematical formula and one should be always go by the rigors of  shallow bi-valued Logic. 
  • If we do not express whatever we know, how will others be able to correct your understanding. Only by discussion with people, you get a good picture of the subject matter.
  • If we are not ready to commit mistakes, you are not going to learn. 
  • Whatever we know, If we express it to others, you are effectively setting a context for further learning and making rectification to our understanding. 
  • The quest for comprehensiveness comes from the fear of being scrutinized.  If you do not allow others to scrutinize ourselves, our understanding can be really shallow
  • Gopi is a "scientist" by psychie and I am an engineer in my approach  
  • Gopi is suffering from a psychological "dis-order" called  Cognitive Closure
I coined the term, "Pre-Web Syndrome" to explain this phenomena as I was typing this blog. Let me describe this phenomena and why people do suffer from it.

Before the advent of the internet and communication mechanisms, people were living in Silos (small towns or villages). They used to know each other well. Life was limited to village square or play ground. 

There was no TV, Facebook, Youtube, even Radio was expensive. Flaunting Knowledge was one of the favorite pastime. People used to debate about various matters and it was a winner take it all kind of situation. This forced people to be secretive and possessive about their knowledge. They were vary of loosing the argument or two, If their knowledge is not comprehensive. In fact, what you know determined your identity.

In the post web world, especially after Web 2.0, Knowledge is just a commodity and race has been on to produce artifacts out of one's Knowledge. We need to reify(materialize) our knowledge into some kind of form which people can consume and profit for themselves. Now, there is an estimate that every eighteen months ( like Moore's law) the knowledge of the world doubles. It is not viable to understand everything. 

Subjects like Physics, Economics,Biology,Mathematics (even sub-disciplines like Differential Geometry) takes years to understand, let alone, master. The pragmatic approach is to understand something and try to leverage it. This necessitates further learning with an objective in mind. Perhaps, prolonged exposure to things will get you a moniker as an expert. 

Let me create a thousand feet definition of the term "Pre-Web Syndrome"

"Pre-Web Syndrome is a psychological state where Individual or group in question is stuck up in a state of affairs where they are fixated on the sacredness of the Knowledge"

Are you a Gopi? Probability of one being a Gopi is near to one. At least, I was a Gopi sometime back.

PS:- I have decided to create a character Gopi to describe the instances of Pre-Web Syndrome. Soon, you can see Gopi's view on various matters.





 

Andrew NG's Machine Learning course is a "Waterloo" for Programmers



















A Quick Note Before you read the main content:- Machine learning course given by Andrew NG ( He is an excellent teacher as well) is the best course out there. But, it is suited only for a small fraction of humanity. I have seen lot of people in my part of the world, who has wasted time, effort and money chasing to master what Andrew has to offer. Unless, an institution (a college,government or your company) funds your study, do not go for it. Your ROI will be very low.

Now a days, the term Data Science is being used  quite often than not. I am also taking keen interest in the topic, as it is important not because of it's purported uses. Because, it has got sizable Demography behind it. Moment Data science is mentioned, people will talk about Data Mining, eventually reach Machine Learning.

Andrew NG is a fantastic teacher and researcher on the topic of Machine Learning. His Stanford lectures and Coursera lectures are available online. Out of the twenty five friends who has attempted his course, only one has managed to finish it.

What is problematic with the course?

  • The Course is bit mathematically challenging. I do not know a single individual (including myself) in my circle, who has got the mathematical inclination or sophistication to fully comprehend the course. Only 1% of the world can really think at the sophistication for the course. 
  • The course is meant to produce Machine Learning researchers. By learning the algorithms, you are not going to use the knowledge as there are smarter people like Andrew who will write better than us and deliver it as a Open Source Library.


  • Because of the presence of excellent Machine Learning software libraries, only few can make a living through it. A programmer's introduction to Machine Learning seems to be a better option than going for  a Mathematician's introduction to Machine Learning. 
What is a good option according to you?

The best way to understand learning algorithms for programmers is read books or take courses meant for programmers.

Three good books I can suggest are 
  • Programming Collective Intelligence by Toby Segran
  • Machine Learning in Action by Peter Harrington
  • Algorithms of the Intelligent Web By Haralambos Marmanis and Dmitry Babenko
Also play with following libraries (any of the two)



  • Apache Lucene and Apache Solr Search system
  • Apache Mahout
  • Weka Machine Learning toolkit
  • skikit-learn (Python) 
After this, refresh your Calculus (Multivariable calculus) and Inferential statistics knowledge. Then, you are in a Position to Andrew. If you go through those books and libraries, you do not need to go to understand Andrew, as you will be able to become a employable commodity.



Tomorrow's Ruby meetup covers the topic of MetaProgramming

Meta-programming is a technique with which you can modify the program itself during it's execution or expand intent of programmers to realize repetitive logic during the compile time. You have to read the previous statement once again to avoid confusion. Normally, programs change data and program code itself is not mutable.


In the case of C++, because of the side effect in the implementation of Templates, there is compile time Meta-programming as well. The C++ world calls this, Template Meta-programming.Technically speaking, C/C++ macro is another way to implement Meta-Programming. You define a Macro and the C Pre-Processor will expand the Macro before the compiler proper gets the code.

The Ruby language has got excellent support for Meta-Programming. In fact, Ruby is being used to create Domain Specific Languages (DSL), as it is one of the best languages out there to implement Internal DSLs. Internal DSLs are DSLs which uses the host language constructs itself to implement Domain specific constructs. You can consult Deabshish Ghosh's book, "DSLs in Action" to understand more about Meta-Programming and DSLs.

Nithin Bekal will be taking a session Meta-Programming at the next Kerala Ruby Users group meeting to be held tomorrow. Based on my interactions with him, he is one of the most informed guy about Programming language techniques. His blog is available here

The details of the venue is available here.

The October meetup on Sunday, the 19th at Whatznear office in Palarivattom.

Address: Blue Finch Online Services Private Limited, First floor, 32/2467, Tagore Lane, Palarivattom Pincode : 682025
Location on maps | RSVP on Google+ | Blog announcement

Friday, October 17, 2014

A veiled threat to convert to Islam from an erstwhile colleague and my thoughts

Since this particular post had some "Ad hominem" lines against an erstwhile colleague of mine, on mutual consent, I have hidden the contents to protect his privacy.

IMHO, she is one of the smartest lady I have heard of

















Raj Kumar is a revered name in Karnataka. If you have seen his movies, you will also like him. Naturally, he swayed Kannada speaking people all across the world. The way Raj Kumar dresses in his movies were impressive,If you judge from the standard of his times.

Prem Nazir, MG Ramachandran, NT Rama Rao and Raj Kumar were ruling the reel world from 1950s to 1985. What is curious about Raj Kumar out this quartet is the fact that he did not show electoral political ambition. ( He was an active supporter of Kannada language advocacy though) He might have had personal ambition like any other person, but resisted or restrained by his dear ones, friends and fans alike.

Couple of Kannadigas have told me that, had it not been for Parvathamma,his wife, he would have plunged into the whole mess. If what my friends told me has got element of truth, she is a smart lady.

One reason why Raj Kumar is widely revered in Karnataka (and other states) is because he has not been criticized for his political affiliations. So, credit should go to Parvathamma. She gave birth to five kids for him.




Wednesday, October 15, 2014

"Speculative Learning" - An "algorithm" to stay alive in the Software Industry

Often people have accused me that my knowledge about things are shallow. The primary reason they tell me is that I dabble with too many things at the same time.

It is a subjective argument, the way I look at it. I do only one or two technology at a time. It so happens that, I do keep track of what I have done. The knowledge acquired for doing new things are also mapped to the things I  have done in the past.. I call it, learning in Retrospect. Most people do learn when there is prospect, I do retrospective learning as well.

Writing random toy programs leveraging programming environments which I can lay hands on does help me to learn new tricks and stay abreast. What I learned for fun, has become source of profit.

When you pick up something, have some kind of potential application in mind. Either, it can be some kind of cross skill acquisition to broaden your portfolio or it can be an exploration to reason about something, we need to speculate about areas where the newly acquired skills can be used. I call this phenomena, "Speculative Learning".

Some recent Anecdotes of the advantages of speculative learning (after 2009)

  • Wrote some DirectX 10 (had worked with DirectX 7) code for fun and it culminated in a contract to work on a graphics project with Direct3D and shaders
  • Played with GNU Linux, wrote some programs to understand the parallel between Wndows and GNU Linux. Did at leat, five projects under GNU Linux using C/C++, Mono, Harbour and Java.
  • Experimental adventure to Objective C using Linux culminated in becoming a MAC OS X/iOS developer.
  • Played with Mono quite a bit and ended up getting  a contract to do Image processing application using Mono/C# for MAC OS X
  • Learned Groovy out of curiosity and worked on one consulting Engagement. This experience helped me to claim that I know "some Spring" ( a Grails generate Spring artifacts for execution) and got an engagement to work in Sri Lanka
  • Took a look at Node.js to read a book on JavaScript. Toyed with the idea of writing a JavaScript backend for my compiler. A company hired me to write a Mobile rule engine which generates Javascript on the fly.
  • Continuous dabbling with Java code helped me to transition myself into JEE, without much trouble. 
  • Learned Apache Lucene, Solr and related libraries. Have won business for my company leveraging that.

For a person with hammer, everything is a Nail!

Each new hammer you are acquiring helps you to nail more things.

Learn Jetty, Embed it to make your Application a Servlet container.

I was exploring options to embed an HTTP server in a java application. I had heard about Jetty in the year 2012, when I had gone to Sri Lanka, for an engagement.

After coming back to India, out of curiosity, I had just played with the server and blogged here. Jetty can be embedded inside your application to make your own custom application an HTTP servlet container.

Jetty server is embedded in Google App Engine, Hadoop,Apache Geronimo and various projects under Apache,Yahoo Simbra and scores of other high visibility platforms.

The information about Jetty is available here and here. You can download the software from here.

I am planning to explore it, further.


Want to know the Political history of Islam?

I just happen to read a very informative article written by one Mr. Tim Urban. His article is centered around the current Iraq and it's conflict. But, he also traces the lineage of Sunni and Shia sects within Islam.  The article is about How the Sunni Shia divide is one of the reasons for the division within Modern Iraq.

He starts from Prophet Mohammed and ends up with ISIS. Rather than me writing about it, you can read it from the original source.


Economics favor sweat, these days

Imagine that, you want to solve an Advanced Mathematical or an Algorithmic problem. How will you go about it? The problem at hand is one which an expert in that particular domain might be able to solve  easily. If we try to solve it ourselves, it might take ages or cannot be solved at all.

You go to a college and ask a professor there to solve it for you and pay him, Rs. 10,000 (200 USD)

How much it will cost you to hire a PHP programmer to write code for  a mundane Business problem?

It will be costing you at least, Rs. 30,000 per month. ( 600 USD).

There are countless instances where cognitive difficulty is not the index for the money you make. It is the importance of a set of tasks, in a context which matters.

You have got shortage of people who can implement things. Their sweat value is more than most intellectually challenging tasks.

With the availability of "cheap" information, the focus has shifted to sweat!