Hafidz Zulkifli

Below article was copied from Solai Murugan's blog . All credits goes to him for the fine work. Copied it over so that I'd know where to find it in the future should I ever forget about it. Error :19: error: value saveAsTextFile is not a member of Array[(String, Int)] arr.saveAsTextFile("hdfs://localhost:9000/sparkhadoop/sp1")  Step to reproduce    val file = sc.textFile("hdfs://master:9000/sparkdata/file2.txt") val counts = file.flatMap(line => line.split(" ")).map(word => (word, 1)).reduceByKey(_ + _) val arr = counts.collect() arr.saveAsTextFile("hdfs://master:9000/sparkhadoop/sp1")  Solution   Error caused on the bolted line above. Its due to storing the array value to the HDFS. In scala for Spark everything should be in RDD (Resilient Distributed datasets). so that scala variable can use Spark realated objects / methos. in this case just convert array into RDD ( replace bolded line by ) sc.makeRDD(arr).sav...

A repository of some of the places I've been to in learning Apache GraphX. Apache Spark ( http://spark.apache.org/docs/latest/ ) How can you not learn about Spark to learn GraphX? A nice introduction to Spark, and from there you can dive into other components. Download Spark, and just start playing with Scala. First Step to Scala ( http://www.artima.com/scalazine/articles/steps.html ) More tutorials on Scala. A step-by-step approach on some basic Scala skills Functional Programming Principles in Scala ( https://class.coursera.org/progfun-004/lecture ) A course on Scala by none other than Martin Odersky himself. A great way to learn Scala and work in Spark better. AMP Camp by Stanford ( http://ampcamp.berkeley.edu/stanford-workshop/index.html ) Exploring the BDAS stack by Stanford University. If you're in the States - this is probably the best way to learn about Spark, Scala, GraphX, MLib. Why do I put that restriction? It's because to be able to make full use o...