Learn Python through public data hacking

http://www.youtube.com/watch?v=RrPZza_vZ3w
http://www.dabeaz.com/pydata/

Scala Tips - List

http://stackoverflow.com/questions/8601041/simplest-way-to-sum-two-lists-in-scala

I want to sum each element in list A with the element in list B, producing a new list.

// Poor

List(1,2).zip(List(5,5)).map(t => t._1 + t._2)

// Great

For two lists:

(List(1,2), List(5,5)).zipped.map(_ + _)

For three lists:

(List(1,2), List(5,5), List(9, 4)).zipped.map(_ + _ + _)

For n lists:

List(List(1, 2), List(5, 5), List(9, 4), List(6, 3)).transpose.map(_.sum)

// Great

you could even improve your snippet to get rid of using the clumsy _1, _2:
List(1,2) zip List(5,5) map { case (a, b) => a + b }

Spark Tutorial

http://mbonaci.github.io/mbo-spark/

Getting started with Spark

This tutorial was written in October 2013.
At the time, the current development version of Spark was 0.9.0.
The tutorial covers Spark setup on Ubuntu 12.04:

• installation of all Spark prerequisites
• Spark build and installation
• basic Spark configuration
• standalone cluster setup (one master and 4 slaves on a single machine)
• running the math.PI approximation job on a standalone cluster

My setup

Before installing Spark:

• Ubuntu 12.04 LTS 32-bit
• OpenJDK 1.6.0_27
• Scala 2.9.3
• Maven 3.0.4
• Python 2.7.3 (you already have this)
• Git 1.7.9.5 (and this, I presume)

[david@david-centos6 spark]$ MASTER=spark://david-centos6:7077 bin/spark-shell