Data Science & Engineering Platform: Data Lineage and Provenance for Apache Spark

-

This is the third in a series of data engineering blogs that we plan to publish. The first blog outlined the data science and data engineering capabilities of Hortonworks Data Platform. Motivation Apache Spark is becoming the de-facto processing framework for all kinds of complex processing including ETL, LOB business data processing and machine learning. […]

The post Data Science & Engineering Platform: Data Lineage and Provenance for Apache Spark appeared first on Hortonworks.

Comments

Post a Comment

Popular posts from this blog

Underwater Autonomous Vehicles Helping Navy Get More for the Money 

-
Image
By AI Trends Staff     As the  US Navy  strives to get to a 355-ship fleet by 2034, part of a 30-year plan the Navy outlined in 2019, it is finding it economical to use underwater autonomous vehicles. They meet many of the requirements of larger submarines at a fraction of the cost.     For example, the Navy awarded Boeing a $43-million contract to build four Orca Extra Large Unmanned Undersea Vehicles (XLUUVs) last year, described in a release from  USNI News .      Boeing based its winning Orca XLUUV design on its Echo Voyager unmanned diesel-electric submersible. The 51-foot-long submersible is launched from a pier and can operate autonomously while sailing up to 6,500 nautical miles without being connected to a manned mother ship, according to the Navy.   In military spending terms, the versatility of the Orca at the price is “fairly unheard of in military spending,” according to an account in  The National Interest . The nearest equivalent was cited as the Navy’s Littoral Com
Read more

Canada regulator seeks information from public on Rogers-Shaw deal

-
(Reuters) - A Canadian regulator on Tuesday asked for information from the public on telecom company Rogers Communications Inc's planned C$20 billion ($16 billion) purchase of Shaw Communications Inc to check for competition concerns. The deal announced in March to create Canada's second-largest cellular and cable operator has attracted regulatory scrutiny. The Competition Bureau said in August it obtained court orders to advance its review. The regulator on Tuesday asked market participants and Canadians to come forth with information that could assist the regulator's probe on whether the deal will reduce or prevent competition. The Competition Bureau said its review was ongoing and no conclusions have been made. ... Read More on Datafloq
Read more