Hafidz Zulkifli

This post marks my first attempt in trying to force myself to gain a better understanding of the books that I've read. Previously, I find myself reading books after books without being able to recall the important things that I've learned earlier. It's rather frustrating to be honest. So I'm trying this out as a way for me to push myself to understand the book and synthesize the various concept and ideas that are conveyed from the book. A disclaimer: My reviews will not attempt to be neutral or unbiased - as I feel that any attempt for me to try and write such kind of a blog post would result in a dry and boring outcome. Guess you could say that it'd probably be much more of a rant rather than review. Moving on. I bought the book from Amazon quite awhile back in April and it has been on the shelf for quite sometime as I was another book at that time. The outline is rather interesting, as it highlights the pitfalls of big data implementation from a f...

The Sexiest Job - HBR I've talked about this subject in some of my posts in my earlier years of working as a data scientist, namely in these 2 blog posts: 1.  Journey In Data Science 2.  Hindsight, 8 Months Down The Analytics Road So now 3 years down the road, I guess I am a little more knowledgeable on the matter, a little bit wiser. Back to that definition I was talking about, recently there has been two articles which I think provides a good description of what are the skills needed to become a data scientist, and what are the role that a data scientist play in a day to day setting. In the final half of this post, I'll include my 2 cents on the articles and how it relates to my daily work. The Skills [1] Picking it up from Forbes (which in turn picked it up from Quora), the top 5 skills are: 1. Programming.  I guess this is pretty much a no brainer. Programming skills do come in handy especially when you're trying to (1) massage data, and (2) auto...

A lengthy title. I know. Cutting to the chase, I needed to transfer data from my RDBMS to a Hadoop cluster and I only had access to Hue/Oozie (since I’m not an admin). I knew that I could use Sqoop to do it — but I’ve never really done it before. It was freaking hard/annoying! So to help others out there who might be in a similar predicament as I was, here are some 101. 1. I assume you already know how to use the Workflow Editor, so from there, create a new Workflow. 2.  Drag a Sqoop action from the panel above and click OK. 3. You’ll get some pre-filled sqoop command in there which you can use as reference. Hop to Apache Sqoop to learn more about all available arguments you can use. 4. There’s 2 way you can go about entering the Sqoop command from here on out. You can either type in the Sqoop command in the text box, OR if you’re thinking of using a query in your command, my recommendation is to use the argument window. The latter is based on a post ...

The recent outbreak of the Ebola virus disease (EVD) has shown how quickly diseases can spread in human populations. This threat is, of course, not limited to EVD; there are many pathogens, such as various types of influenza (H5N1, H7N9, etc.) with the potential to cause a pandemic. Therefore, mathematical modeling of the transmission pathways becomes ever more important. Health officials need to make decisions as to how to counter the threat. There are a large number of scientific publications on the subject, such as the recent Science publication by Dirk Brockmann, which is available here. Professor Brockmann also produced videos to illustrate the research, which can be found on YouTube (video1, video2, video3). It would be interesting to reproduce some of the results from that paper and generally explore the subject with Mathematica. Full article here:  Modeling a Pandemic like Ebola with the Wolfram Language

For the past few days I've been trying to gather coordinates of places around KL, in the attempt to tag those places to our cell towers for a more in depth market analysis of our consumers. While one could in a sense get those coordinate manually from Google Maps, and jot down the coordinates on a spreadsheet - it seemed like "not-so-smart" solution and troublesome to do in the long run (ie. your boss asks you to find the number of customers that's visiting a particular shopping mall today, and tomorrow he wants to know the numbers of customers that visits golf courses in the outskirt of town. In such case jotting down the coordinates manually would be very time consuming  - not to mention crazy) Initial attempts include web scraping and looking into the source code of the map (more on that topic later). Somehow rather those didn't work well as expected. Hence now I turn to Google API, or more specifically - Places. There are actually a lot of stuff that ...

Working in a team of data science enthusiast can have it's benefits. In our team of a few, none of us can claim we're good in data science and/or big data analytics as we're pretty much new in this field. Data scientists are hard to come by, and most articles out there can attest to that. Accenture in this suggested a nice idea in their article " The Team Solution to the Data Scientist Shortage ", if a data scientist person is hard to find - why not have/build a team that has the necessary skills of a "data scientist"? Inspired by this, I've set myself a goal to at least master some of the necessary skills that make up a data science guy. 8 skills of a Data Scientist, from the Accenture article mentioned above. Which makes sense really. To be a master of all the above mentioned area would consume an insane amount of time. Thus to be able to segregate the task around and focus on achieving the end goal - together - as a team; would mean ...