Hafidz Zulkifli

Lately I've been studying Shiny and how to use in R. It's a cool arsenal to have while using R as you can really quickly develop a data product right from R itself. Of course you could say load the data up in tools like Tableau or Qlik Sense are have a much cooler/sexier visualisation - but that's not the point I'm trying to bring here. For a quick preview of what I managed to conjure up with Shiny, pop over to Social Network  . Couldn't help it - it just had to be a network graph - again :P. A real sucker for graphs I am. Anyways, what it aims to demonstrate are how from a social network graph like that, you can derive the centralities (degree, closeness etc) and from there - the roles of each nodes based on how they are connected to each other. I'll not be making the claim that it's correct in any way - it's just something that I've picked up from Drew Conway, based on his presentation on Socio-Terrorism . For more details - check ou...

A repository of some of the places I've been to in learning Apache GraphX. Apache Spark ( http://spark.apache.org/docs/latest/ ) How can you not learn about Spark to learn GraphX? A nice introduction to Spark, and from there you can dive into other components. Download Spark, and just start playing with Scala. First Step to Scala ( http://www.artima.com/scalazine/articles/steps.html ) More tutorials on Scala. A step-by-step approach on some basic Scala skills Functional Programming Principles in Scala ( https://class.coursera.org/progfun-004/lecture ) A course on Scala by none other than Martin Odersky himself. A great way to learn Scala and work in Spark better. AMP Camp by Stanford ( http://ampcamp.berkeley.edu/stanford-workshop/index.html ) Exploring the BDAS stack by Stanford University. If you're in the States - this is probably the best way to learn about Spark, Scala, GraphX, MLib. Why do I put that restriction? It's because to be able to make full use o...

Can Netezza do network analysis? That is the current predicament that I was put under for the past couple of weeks. Based on the set of hardware that I've (read: my company) got with me - I'm supposed to be using it to calculate the usual set of social network measurements (i.e degree, betweenness, closeness, eigenvector etc).  2 weeks have passed and while it was relatively easy to calculate the degree centrality - betweenness have proven to be quite a challenge. So far I've been able to translate Djikstra's work in determining shortest path between nodes, and using those to determine the betweenness (refer back to the formula of betweenness centrality if you're lost here). The results have been tested on a small scale network with 10 vertices - and the values match with ones given in Gephi - so initially I was quite confident to be able to simply pump in the actual data from my telecom network. That however didn't go as smoothly. The amount of memo...

In Part 1, I wrote mostly about how I ended up working in SNA and I touched a bit on how Gephi can be used as the simplest solution to visualize your network. If you're good in JavaScript - the source generated by Gephi would've given you some idea on how to expand it further to have more selection panels and filtering capabilities. Unfortunately for yours truly - I suck at it.  Of course while I could spend some of my precious time learning a new language, I guess I could also make use of some ready-made tools available in the market. I chose Qlikview in this case. Mostly because it has a free personal license (who doesn't like free stuff right?)  Qlikview Traditionally - Qlikview doesn't support network graph visualization in any of it's out-of-the-box widgets. However, the beauty/strength of Qlikview in my opinion is it's support for extended plugins - which they call extensions. Using JavaScript, one has the ability to create any sor...

An idea is worthless unless implemented right? You can have all the data in the world but if you can't articulate it or visualize it for others to understand, then it's really just another data in your data warehouse. The last couple of months me and my colleagues were working on social network. Basically trying to understand how the subscribers are interconnected and identifying who are the influencers for targeted marketing - or so we thought that's how it should be. Business justification aside - it was an interesting topic to dive into. After a few days, we managed to come up with our edge and node list, and later ran a few centrality algorithms to measure each individual within that network. Specifically, we were measuring: Degree - The number of direct connections that one has. ie How many direct friends does he have? Closeness centrality - How close (by means of hop) is a person to each of the person in their network? Between-ness centrality -  Identifying w...