Cargill, of suburban Minneapolis, represents a formidable new competitor. Its $134.9 billion in fiscal 2014 sales ranked it as the largest U.S. agricultural firm and the country's largest privately held company. The 149-year-old company has long advised farmers on farming strategies and the best time to sell grain. In 1996, Cargill began sampling soil and experimenting with applying different amounts of fertilizer to various fields, depending on how many nutrients the ground already held, Mr. Becraft said.
NextField DataRx represents a more information-intensive version of Cargill's advisory service, incorporating historical weather data, satellite imagery and farmers' own information.
The animation above shows all scheduled flights over a 24h period (based on 2008 data). Every day 93,000 flights are starting from approx. 9,000 airports. At any time there are between 8,000 and 13,000 airplanes in the air. This animation was produced to be shown on the high definition 3D-Globe "Orbitarium" in Technorama - The Swiss Science Center in collaboration with Institute of Applied Information Technology InIT, Zurich University of Applied Sciences, Winterthur.
Now, think a few years from now where most planes will have sensors in engines, flaps, landing gear and how much performance/maintenance data they will generate. Boeing 787s are expected to generate 1/2 terabyte of data a flight.
Think how much data will be streamed to and from these planes as more airlines offer wi-fi at 12mbps and better speeds (as ViaSat is delivering on some Jetblue flights)
Or how many page views will FlightAware get as more families and passengers track flights (below is image from their Live Flight Tracker)
and depressingly think how much more emissions will go towards global warming – half a pound of CO2 per passenger mile.
It’s an irony of the second Age of Reason that the abundance of data—the effervescence of sources and ease of delivery—makes so many more questions answerable while at the same time making it very easy to get lost. We’ve dedicated an issue to exploration, to a broad, cross-platform look at the fruits of Big Data.
Employers say the ideal candidate must have more than traditional market-research skills: the ability to find patterns in millions of pieces of data streaming in from different sources, to infer from those patterns how customers behave and to write statistical models that pinpoint behavioral triggers.
At e-commerce site operator Etsy Inc., for instance, a biostatistics Ph.D. who spent years mining medical records for early signs of breast cancer now writes statistical models to figure out the terms people use when they search Etsy for a new fashion they saw on the street.
To get help, employers are increasingly looking to an elite program called Insight Data Science Fellows Program, which helps funnel doctoral candidates from fields like astrophysics, neuroscience and math into the profession. The program, based near Stanford University and funded by tech companies, has a 100% placement rate.
The Chicago mapping and traffic information enterprise started life as NAVTEQ, acquired by Nokia in 2007. The map databases curated here go into four out of five cars with in-dash navigation, and the company makes 2.7 million map database revisions every day. Here's global traffic-monitoring effort supports nav/traffic routing in 41 countries and processes more than 1 billion data points per day coming in primarily from anonymized cellphone GPS signals and roadside traffic monitors. Twenty-five traffic "editors" cover North America and Australia from Chicago, monitoring police scanners, government Twitter feeds, and 12,000 traffic cameras to provide real-time traffic route guidance.
The traffic team is done with this probe data minutes after it arrives, but it doesn't get discarded. The Big Data analysis team at Here teases other useful info out of it, such as mapping drive-through restaurant lanes and confirming POI viability. (No cellphones have stopped at that supposed fuel station in a month; should we strike it from the database?) But the most interesting knowledge they're teasing out of this pile of ones and zeroes is behavioral. By studying the speed traces of millions of vehicles on freeway ramps, dead-man's-curves, and blind-uncontrolled intersections, they can begin to model how real humans behave in these situations and teach this to the would-be autopilots.
Companies ranging from established giants such as IBM, SAS, and Microsoft to startups such as Tranzlogic and Kaggle offer affordable, cloud-based data-crunching services-which can help you get nondigitized data into data-crunchable form-and today virtually anyone can get his or her hands dirty in the great Big Data mud pile.
Businesses successfully mining Big Data are cross-referencing their internal information-pricing histories, customer traffic patterns-with multiple outside sources to increase revenue by understanding customers' behavior better, reducing costs by eliminating inefficiencies and human bias, strengthening client bonds by anticipating clients' needs, enriching service offerings with new knowledge, and giving employees new tools to perform their jobs better.
Oracle invited me to moderate a panel in their DaaS launch today. Watch the whole hour for some great perspectives on this exciting new market category, and our panel starting around 26.00.
Talking to Steve Miranda and Omar Tawakol before and after I got even more excited about the concept
a) There is a new generation of data asset entrepreneurs like Omar (whose company, BlueKai Oracle acquired) who are about separating signal from noise– under NDA I have been briefed by other entrepreneurs focused on supply chain, HR and other DaaS. Scary smart folks.
b) Watching Omar talk about all the social and other marketing data feeds, Steve talking about feeds from machines and wearables, you realize we are not anymore in the Kansas of internal, structured data so many companies are still heavily invested in
c) How rapidly business and deployment models are changing. While the old model of “buy hardware, install software, load data, clean for quality – pay for all that” is not going away the speed of getting started with and the economics of DaaS are going to be compelling
d) How effortlessly Oracle with its decades of data management experience and horizontal and vertical application knowledge could play in a wide range of DaaS categories.
Each month more than 11 million people–mostly 35- to 65-year-old women–visit Wayfair.com to browse its massive housewares catalog, an online directory hundreds of times larger than any Sears, Roebuck ever produced. Shipping is free for orders over $49; assembly is usually up to you. Wayfair doesn’t make anything. Many of its goods are produced by mom-and-pop operations, and the site will carry a product even if it sells it only once.
The key to this enterprise is a series of algorithms that fulfills orders–with a 98% success rate that’s improving all the time. Deployed to manage 7,000 vendors and a head-spinningly convoluted supply chain, that secret sauce makes shopping a virtually frictionless experience. Wayfair is as much a data miner as it is a retailer.
As many as 1 in 10 patients respond well in clinical trials of experimental medicines that U.S. regulators end up rejecting, according to the National Cancer Institute (NCI). To understand why these patients had such a response, researchers are beginning to use DNA sequencing technology to determine if the patients they call “exceptional responders” carry gene variations that can lead to better targeted therapies, including new treatments and the reconsideration of others.
Traditional treatments such as chemotherapy kill healthy cells along with malignant ones, but targeted therapies are designed to leave healthy cells unscathed and home in on cancer cells that make tumors grow and spread. The catch is that they don’t work for everyone, and even patients who find them helpful tend to develop resistance over time. The NCI and academic medical centers including Memorial Sloan Kettering Cancer Center in New York, the Dana-Farber Cancer Institute and Massachusetts General Hospital in Boston, and the Broad Institute in Cambridge, Mass., are creating a national database of exceptional responders to aid research. “What was yesterday’s miracle event is today becoming a subject of scientific inquiry,” says Leonard Lichtenfeld, an oncologist and the deputy chief medical officer of the American Cancer Society.