Pivot Tables

In the previous lesson we cleaned up the data we’ve been collecting. Now the question is: what can we do with it? Look at this table. It was created from the over 65,000 rows of data in the movie rating dataset we saw a few lessons ago….

	Women		Men
	Number	Avg. Rating	Number	Avg Rating
All Movies	16,716	3.54	48,819	3.53
Star Wars	102	4.23	284	4.37
Abyss, The	20	4.00	82	3.55

How long do you think it would take you to calculate the values in this table from the raw dataset of ~65,000 rows? Justify your answer.

Why make visualizations?

Yesterday you looked at a bunch of data from the Pew Research Center that was all presented visually in graphs and charts. The question is: why? Why did they choose to make a bunch of charts and graphs rather than just showing the raw data itself?

• “Why did Pew Research choose to make a bunch of charts and graphs rather than just showing the raw data itself?“
• “List a few advantages and disadvantages (at least 2 for each) of using visualizations to communicate data”

Google Flu Trends

Read on of the following articles and answer the following question:
“Why did Google Flu Trends eventually fail? What assumptions did they make about their data or their model that ultimately proved not to be true?“

Wired - What Can We Learn from the Epic Failure of Google Flu Trends?
NYTimes - Google Flu Trends: The Limits of Big Data
Nature - When Google got flu wrong
Time - Google’s Flu Project Shows the Failings of Big Data
Harvard Business Review - Google Flu Trends’ Failure Shows Good Data > Big Data

Trends

Last week we started to collect data about ourselves so that we could learn about trends and patterns in our behavior. What is meant by the term "trending"?

Data Collection

Data is all around us. What are some examples of data and give a couple specific reasons as to why someone would collect data.

Intro to Data

The last project you did (Encoding an Experience) was about organizing and structuring digital data to represent complex information. You did it by thinking about bits. In reality we typically don’t have to break digital data down all the way to bits in order to work with it, but understanding that digital data at its root is just bits gives you insights into working with larger data sets. We are about to embark on a new series of lessons where you will work with real data sets and learn how to use to tools to explore and extract information and knowledge from the data. 

But before we do that, what exactly is data? Explain in your own words.

Encode an Experience.

We have come to the end of another unit where we have been building layers of encodings on top of the foundation of bits.


First we learned to develop binary numbers, then ASCII text, then formatted text, and finally color images. High-level encodings are actually quite removed from the underlying bits from which they are made.

What does it mean to encode something?

When we use the term abstraction, what are we referring to?