Learning GraphX

A repository of some of the places I've been to in learning Apache GraphX.

Apache Spark (http://spark.apache.org/docs/latest/)
How can you not learn about Spark to learn GraphX? A nice introduction to Spark, and from there you can dive into other components. Download Spark, and just start playing with Scala.

First Step to Scala (http://www.artima.com/scalazine/articles/steps.html)
More tutorials on Scala. A step-by-step approach on some basic Scala skills

Functional Programming Principles in Scala (https://class.coursera.org/progfun-004/lecture)
A course on Scala by none other than Martin Odersky himself. A great way to learn Scala and work in Spark better.

AMP Camp by Stanford (http://ampcamp.berkeley.edu/stanford-workshop/index.html)
Exploring the BDAS stack by Stanford University. If you're in the States - this is probably the best way to learn about Spark, Scala, GraphX, MLib. Why do I put that restriction? It's because to be able to make full use of the workshop - you'll need to set up an Amazon EC2 account, which is probably region based. That being said - I haven't really tried it yet. So if you're in Asia and you managed to go through the whole workshop - let me know!:)

Sotera (http://sotera.github.io/distributed-graph-analytics/)
Their Github contain a few SNA algorithms developed by them in Giraph and GraphX. Apart from sharing the source files - they also provide you with an image for you to explore their algorithms in more detail. So if you're not able to play with Stanford's cluster above, this is probably the next best thing (aside from getting a hands-on on a real SNA algorithm)

Spart Summit (http://spark.apache.org/documentation.html)
Documentations, videos, training materials, exercises etc. Awesome materials to get you started and for the more advanced!

Comments

Post a Comment

Popular posts from this blog

HIVE: Both Left and Right Aliases Encountered in Join

Recently was stuck on this error while trying to do a JOIN in HIVE. Below is my SQL create table ma.internet_v2 as select A.msisdn, A.imsi, A.dt, A.start_time, A.end_time, A.url, A.ttl_connection_dur_ms, A.ttl_upload_bytes, A.ttl_download_bytes, A.ttl_cdr_cnt, coalesce(B.domain_desc,A.domain_desc) domain_desc, coalesce(B.subdomain_desc, A.subdomain_desc) subdomain_desc from db.internet A left outer join hfz_domain_name_mapping B on instr(A.url, B.url_pattern) > 0; HIVE then give me and error message "Both Left and Right Aliases Encountered in Join". After going through some mail threads it seems that HIVE can support certain types of join. Only equality joins, outer joins, and left semi joins are supported in Hive. Hive does not support join conditions that are not equality conditions as it is very difficult to express such conditions as a map/reduce job. A poster from  Stack Overflow  suggested that one should use WHERE instead of ON. Duly following his advice,
Keep reading

A Basic Recipe for Machine Learning

Image
Ever since wrapping up the three Deep Learning courses by Andrew Ng I've been meaning to write down some of the gems that he's highlighted throughout the course. One of the nice ones that I felt needed to be written down is his general recipe to approaching a deep learning algorithm/model. I've basically summarized it in a flowchart below (because everybody loves a flowchart right?) Basic deep learning recipe What is bias and variance? The below diagram is the typical explanation that I'm sure most of us are used to.  Graphical illustration of bias and variance How can we know if we have high bias or high variance?  For high bias, we could take a look at the training set performance. A poor performance is an indicator of a poor model fit, and signals that we could try to apply a bigger network to get a better fit of the model. For high variance, we could take a look at the validation set (or dev set - as Andrew calls it) performance.
Keep reading

Changing Windows Account password using Outlook

http://theessentialexchange.com/blogs/michael/archive/2007/11/24/changing-user-passwords-with-outlook-2003-2007.aspx . 1) Call the helpdesk - the most likely scenario if you are not in a hosted environment 2) Use the control panel - most Hosted Exchange providers provide a function on a user's control panel enabling the user to change their password, using the functionality in item (3). Most large companies will implement a similar capability for their users. Small companies will likely need to implement item (3) directly or their helpdesk will do it (as in item 1). 3) Enable IISADMPWD - use KB 555071 to enable the use of a web page to allow changing the password of an active directory user.
Keep reading