Academic year 2013–2014
"The success of companies like Google, Facebook, Amazon, and Netflix, not to mention Wall Street firms and industries from manufacturing and retail to healthcare, is increasingly driven by better tools for extracting meaning from very large quantities of data. 'Data Scientist' is now the hottest job title in Silicon Valley." – Tim O'Reilly
- Data Scientist: The Sexiest Job of the 21st Century (pdf)
- Find true love with data mining
The course will develop algorithms and statistical techniques for data analysis and mining, with emphasis on massive data sets such as large network data. It will cover the main theoretical and practical aspects behind data mining.
The goal of the course is twofold. First, it will present the main theory behind the analysis of data. Second, it will be hands-on and at the end students will become familiar with various state-of-the-art tools and techniques for analyzing data. We will use Python for downloading data and implementing various algorithms using its rich libraries and Hadoop's MapReduce framework for mining of large-scale data.
The sixth homework is out; check the homework for the due date.
The fifth homework is out; it is due on June 8.
You can find information about the project at the project web page.
The fourth homework is out; it is due on May 25.
The due date for homework 3 has been extended to May 11.
The third homework is out; it is due on May 4.
The second homework is out; it is due on April 27.
There will be no classes on March 31 and on April 2.
On March 26 we will have lecture in room A2.
There will be no class on Monday, March 10.
The first homework is out; it is due on March 23.
The rooms for the lectures have changed.
The first lecture will be on Monday, February 24.
Aris Anagnostopoulos, Sapienza University of Rome.
When and where:
Monday 10.10–13.30, Room A7
Wednesday 15.45–17.15, Room A5
You can use the office hours for any question regarding the class material, past or current homeworks, general questions on data mining, the meaning of life, pretty much anything. Send an email to the instructor for arrangement.
Chapters for which no book is mentioned refer to the "Mining of Massive Datasets" (see below).
|February 24||Introduction to data mining
Introduction to probability
|Chapters 1.1, 1.3
Introduction to data mining
A crash course on discrete probability
Check the background probability chapters below
|February 26||Introduction to probability (continue)|
|March 3||Introduction to probability (continue)
An overview of Python
|Check the Python resources below|
|March 5||Overview of information retrieval, part 1||IIR book, chapters 1.0–1.1, 2.0–2.2|
|March 12||Overview of information retrieval, part 2||IIR book, chapters 1.2–1.3, 4.2|
|March 17||Overview of information retrieval, part 3
Introduction to MapReduce and Hadoop, part 1
|IIR book, chapters 6.2–6.3, 7.1.0
Jeff Dean's talk on Google's distributed systems
Check the "Hadoop resources" below
|March 19||Introduction to MapReduce and Hadoop, part 2||Chapters 2.0–2.3.1
Check the "Examples done in class" section below.
|March 24||Similarity and distance measures||Chapter 3.5, Wikipedia on LCS|
|March 26||Homework solutions|
|April 7||Shingling and minwise hashing||Chapters 3.0–3.3|
|April 9||LSH for documents||Chapter 3.4|
|April 14||Mining frequent itemsets, part 1||Chapters 6.0–6.2, 6.3.1|
|April 16||Mining frequent itemsets, part 2||Chapters 6.4.0–6.4.4|
|April 28||Classification, Naive Bayes classification, Clustering, K-means||IIR book, chapters 13.0–13.3
Chapters 7.0–7.1.2, 7.2–7.2.3, 7.3–7.3.3
Chapter on k-means of the book of Christopher M. Bishop
|April 30||Modeling, maximum likelihood||Notes|
|May 5||Soft clustering, and expectation–maximization
Some Concepts of statistics
|Slides on statistics by B. Liseo (in italian)
Slides on linear regression by B. Liseo
|May 7||Social networks and structural properties||Slides on social networks|
|May 12||Link analysis, PageRank, Personalized PageRank||Chapters 5.0–5.1, 5.3, 5.4.4|
|May 14||Streaming algorithms, part 1||Chapters 4.0–4.2, 4.5.5|
|May 19||Streaming algorithms, part 2||Chapters 4.3–4.5|
|May 26||Dimensionality reduction, SVD, Recommendation systems||Chapters 7.1.3, 9.0–9.3, 11.3.0–11.3.3, 11.3.5|
Check the "Examination format" section below for information about collaborating, being late, and so on.
Handing in: You must hand in the homeworks by the due date and time by an email to the instructor that will contain as attachment (not links!) a .zip or .tar.gz file with all your answers and subject
[Data Mining class] Homework #
where # is the homework number. After you submit, you will receive an acknowledgement email that your homework has been received and at what date and time. If you have not received an acknowledgement email within 1 day after you submit then contact the instructor.
The solutions for the theoretical exercises must contain your answers either typed up or hand written clearly and scanned.
The solutions for the programming assignments must contain the source code, instructions to run it, and the output generated (to the screen or to files).
We will not post the solutions online, but we will present them in class.
- Homework 1 (due: 23/3/2014, 23.59)
- Homework 2 (due: 27/4/2014, 23.59)
- Homework 3 (updated 30/4/2014, due: 11/5/2014, 23.59)
- Homework 4 (due: 25/5/2014, 23.59)
- Homework 5 (due: 8/6/2014, 23.59)
- Homework 6 (due: check the homework)
Before the exam you have to implement a project, to which you will also be tested. You can find more information at the project web page.
Textbook and references
The main textbook is the "Mining of Massive Datasets," by A. Rajaraman, J. Leskovec, and J. D. Ullman. The printed version has been updated and you can download the latest version (currently 2.1) from the book's web site.
In addition, we will cover material from various sources, which we will post online as the course proceeds.
The main programming language that we will use in the course is Python. There are currently two main versions of Python, version 2.7 and version 3.3, which are slightly incompatible, but it is easy to translate programs from one version to the other. Even though version 3.3 is newer and better designed, version 2.7 is still more widely used and most available libraries are written for it. Therefore, in the class we will use Python 2.7.
To learn the language you can find a lot of material online. You can start from Python's documentation site: http://docs.python.org/2.7.
If you would like to buy some books, you can check the
- "Learning Python, 5th edition," by Mark Lutz. It is a bit verbose, but it presents well the features of the language.
- "Python Pocket Reference, 4th edition," by Mark Lutz (5th edition expected soon). It is usefull as a quick reference if you know more or less the language and you are searching for some information.
We will use several libraries in the class. The Anaconda distribution has packaged all of them together and you can download it for free.
If you have problems with Python installation you can obtain an ubuntu virtual machine with Python preinstalled. Contact the instructor for more information.
Hadoop and MapReduce resources
For MapReduce and Hadoop you can find a lot of material online, whereas, if you prefer a book, we recommend:
- "Hadoop: The Definitive Guide, 3rd edition," by Tom White. It covers various aspects of Hadoop, including a lot of material that we will not need for the course. A good resource to learn the more advance material.
- "Hadoop in Action," by Chuck Lam. Better organized and with more introductory material. It covers though fewer aspects of the more advance material.
- There is a variety of releases for Hadoop; we recommend to install the 2.2.0 release.
- Here is a tutorial for setting up a pseudo-distributed, single-node Hadoop cluster. If you have a 64-bit operating system check also here. Yet you may need to google around to fix other problems that may pop up.
- With this introductory tutorial you can start learning how to use Hadoop.
- There exists a newer API for Hadoop, which you are recommended to user. Unfortunately the API is written for the old one. You can find the word-counting program here. Also you can find the main differences between the old and the new API here and here.
- For more details for the various classes, the consult Hadoop's Javadoc documentation.
Notes, slides, and other material
Book chapters and notes:
30/4/2014: Notes on modeling, maximum likelihood, soft clustering, and expectation maximization
24/2/2014: Introduction to data mining
26/2/2014: A crash course on discrete probability
5/5/2014: Slides on statistics by B. Liseo (in italian)
5/5/2014: Slides on linear regression by B. Liseo
7/5/2014: Introduction to social networks
5/3/2014: A good introductory source for a lot of the information-retrieval issues that we discussed is the book "Introduction to Information Retrieval," by Manning, Raghavan, and Schütze.
Examples done in class:
19/3/2014: MapReduce and Hadoop
- Count the number of IP accesses at an Apache log: CountIPs.java. You can find an example file here. This a slightly simplified version of the word-count example available at the following tutorial. You are encouraged to go through the entire tutorial. Check also the resources above for Hadoop.
The evaluation will consist of a series of homeworks, a final project, and a final exam. In addition, we will take into account participation during class.
- Every two weeks you will have to work on a set of homeworks, which will include both theoretical and programming exercises. The theoretical exercises will ask you questions that will cover or extend the material that we have done in the class. The programming exercises will ask you to implement some of the algorithms that we have covered in class or to extend the ideas, using either Python, or Hadoop. For instance, we may ask you to download and cluster some twitter posts. To be able to program the solutions you will need to study by yourself programming and programming libraries beyond what we will have covered in the class.
Late policy: Every homework must be returned by the due date. Homeworks that are late will lose 10% of the grade if they are up to 1 day (24h) late, 20% if they are 2 days late, 30% if they are 3 days late, and they will receive no credit if they are late for more than 3 days. However, you have a bonus of 3 10 late days, which you can distribute as you wish among all the homeworks. The homeworks will be discussed and graded at the end, during the final exam.
- Towards the end of the course you will have to select a project and program a solution. You may propose a topic that you would like to work on. More details will follow.
- The final exam will include answering questions about the class material, similar to the theoretical homework questions. It will also include examination of your homework solutions and of the final project, for which you should be able to provide details and prove that you have understood and remember your solutions and your programs.
Collaboration policy (read carefully!): You can discuss with other students of the course about the homeworks. However, you must understand well your solutions and the final writeup must be yours and written in isolation. In addition, even though you may discuss about how you could implement an algorithm, what type of libraries to use, and so on, the final code must be yours. You may also consult the internet for information, as long as it does not reveal the solution. If a question asks you to design and implement an algorithm for a problem, it's fine if you find information about how to resolve a problem with character encoding, for example, but it is not fine if you search for the code or the algorithm for the problem you are being asked. For the final project, you can talk with other students of the course about questions on the programming language, libraries, some API issue, and so on, but both the solutions and the programming must be yours. If we find out that you have violated the policy and you have copied in any way you will automatically fail. If you have any doubts about whether something is allowed or not, ask the instructor.
We expect students to do all the homeworks and have a regular exam (see above). Students who for whatever reason (work, medical reasons, grandmother who died, laziness, etc.) do not do all the homeworks, will have a much longer final exam, in which they must demonstrate that they have knowledge of all the class material and that they are able to implement correctly algorithms such as those asked for the homeworks. In other words, we will ask them during the exam to come up with and implement algorithms similar to those asked at the homeworks.