Quote/Unix has a uniform interface

From PKC
Revision as of 10:38, 16 July 2022 by Benkoo (talk | contribs) (Created page with "{{Blockquote |text=In the Unix world, the unifrom interface that allows one program to be composed with another is files and pipes; in MapReduce, that interface is a distribut...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

In the Unix world, the unifrom interface that allows one program to be composed with another is files and pipes; in MapReduce, that interface is a distributed filesystem. We saw that dataflow engines add their own pipe-like data transport mechanisms to avoid materializing intermediate state to the distributed filesystem, but the initial input and final output of a job is still usually HDFS.

— Martin Kleppmannref>Kleppmann, Martin (Mar 1, 2017). Designing Data-Intensive Applications:The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. local page: O'reily Press.  , Chapter 10: Batch Processing, Page 429</ref>

References


Related Pages

Authored by:Martin Kleppmann


Retrieved from "https://pkc.pub/index.php?title=Quote/Unix_has_a_uniform_interface&oldid=33470"