Personally I despise hubs and changing planes, but you have to admire the algorithms and Big Data of gate, flight, passenger, crew, ground staff. weather and other information that is going into this “peak scheduling”
“Peak scheduling packs planes better because it creates more possible itineraries. Under American's old schedule, a flight from Columbus, Ohio, to Miami might have had 20 possible connecting flights. After the Aug. 19 re-peaking it may have 45. That means more bookings on the Columbus flight, and more people on the connecting flights.
In Miami on a typical weekday, 42 flights depart between 9 and 10 a.m. Then between 10 and 11 a.m., only a handful are scheduled to take off. The process repeats during the day with 10 "banks" of flights that fill about 45 gates at a time.”
Algorithmia is a marketplace where companies can buy small pieces of code or whole programs created by academics, ranging from language-recognition functions to analytics for Web traffic or predicting user purchases.
Deep in the bowels of the Stata Center on the Massachusetts Institute of Technology's campus is an energy war room.
A row of flat-screen monitors lines one wall, showing exhaustive data on energy use in dozens of buildings across the campus. Buildings are displayed in colors that depend on their overall energy use. If a building is red, that indicates an energy leak in one of its lighting, climate-control or ventilation systems, or a water leak. The system, using software from KGS Buildings LLC, can also predict where problems will crop up.
"It makes us more efficient, because we know what to look for," says Balby Etienne, an MIT buildings-systems analyst. He also credits the software for a big drop in temperature and humidity complaints.
Cargill, of suburban Minneapolis, represents a formidable new competitor. Its $134.9 billion in fiscal 2014 sales ranked it as the largest U.S. agricultural firm and the country's largest privately held company. The 149-year-old company has long advised farmers on farming strategies and the best time to sell grain. In 1996, Cargill began sampling soil and experimenting with applying different amounts of fertilizer to various fields, depending on how many nutrients the ground already held, Mr. Becraft said.
NextField DataRx represents a more information-intensive version of Cargill's advisory service, incorporating historical weather data, satellite imagery and farmers' own information.
The animation above shows all scheduled flights over a 24h period (based on 2008 data). Every day 93,000 flights are starting from approx. 9,000 airports. At any time there are between 8,000 and 13,000 airplanes in the air. This animation was produced to be shown on the high definition 3D-Globe "Orbitarium" in Technorama - The Swiss Science Center in collaboration with Institute of Applied Information Technology InIT, Zurich University of Applied Sciences, Winterthur.
Now, think a few years from now where most planes will have sensors in engines, flaps, landing gear and how much performance/maintenance data they will generate. Boeing 787s are expected to generate 1/2 terabyte of data a flight.
Think how much data will be streamed to and from these planes as more airlines offer wi-fi at 12mbps and better speeds (as ViaSat is delivering on some Jetblue flights)
Or how many page views will FlightAware get as more families and passengers track flights (below is image from their Live Flight Tracker)
and depressingly think how much more emissions will go towards global warming – half a pound of CO2 per passenger mile.
It’s an irony of the second Age of Reason that the abundance of data—the effervescence of sources and ease of delivery—makes so many more questions answerable while at the same time making it very easy to get lost. We’ve dedicated an issue to exploration, to a broad, cross-platform look at the fruits of Big Data.
Employers say the ideal candidate must have more than traditional market-research skills: the ability to find patterns in millions of pieces of data streaming in from different sources, to infer from those patterns how customers behave and to write statistical models that pinpoint behavioral triggers.
At e-commerce site operator Etsy Inc., for instance, a biostatistics Ph.D. who spent years mining medical records for early signs of breast cancer now writes statistical models to figure out the terms people use when they search Etsy for a new fashion they saw on the street.
To get help, employers are increasingly looking to an elite program called Insight Data Science Fellows Program, which helps funnel doctoral candidates from fields like astrophysics, neuroscience and math into the profession. The program, based near Stanford University and funded by tech companies, has a 100% placement rate.
The Chicago mapping and traffic information enterprise started life as NAVTEQ, acquired by Nokia in 2007. The map databases curated here go into four out of five cars with in-dash navigation, and the company makes 2.7 million map database revisions every day. Here's global traffic-monitoring effort supports nav/traffic routing in 41 countries and processes more than 1 billion data points per day coming in primarily from anonymized cellphone GPS signals and roadside traffic monitors. Twenty-five traffic "editors" cover North America and Australia from Chicago, monitoring police scanners, government Twitter feeds, and 12,000 traffic cameras to provide real-time traffic route guidance.
The traffic team is done with this probe data minutes after it arrives, but it doesn't get discarded. The Big Data analysis team at Here teases other useful info out of it, such as mapping drive-through restaurant lanes and confirming POI viability. (No cellphones have stopped at that supposed fuel station in a month; should we strike it from the database?) But the most interesting knowledge they're teasing out of this pile of ones and zeroes is behavioral. By studying the speed traces of millions of vehicles on freeway ramps, dead-man's-curves, and blind-uncontrolled intersections, they can begin to model how real humans behave in these situations and teach this to the would-be autopilots.
Companies ranging from established giants such as IBM, SAS, and Microsoft to startups such as Tranzlogic and Kaggle offer affordable, cloud-based data-crunching services-which can help you get nondigitized data into data-crunchable form-and today virtually anyone can get his or her hands dirty in the great Big Data mud pile.
Businesses successfully mining Big Data are cross-referencing their internal information-pricing histories, customer traffic patterns-with multiple outside sources to increase revenue by understanding customers' behavior better, reducing costs by eliminating inefficiencies and human bias, strengthening client bonds by anticipating clients' needs, enriching service offerings with new knowledge, and giving employees new tools to perform their jobs better.