Book/Designing Data-Intensive Applications
Kleppmann, Martin (Mar 1, 2017). Designing Data-Intensive Applications:The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. local page: O'reily Press.
Martin Kleppmann teaches a course on Distributed systems[1]. The author also has a video series called Distributed Systems[2] on Youtube. One may also want to read on Jean Bacon's book on Concurrent Systems[3].
Critical Insights
In the Unix world, the unifrom interface that allows one program to be composed with another is files and pipes; in MapReduce, that interface is a distributed filesystem. We saw that dataflow engines add their own pipe-like data transport mechanisms to avoid materializing intermediate state to the distributed filesystem, but the initial input and final output of a job is still usually HDFS.
References
- ↑ Distributed Systems, Instructor Martin Kleppmann, Website course note
- ↑ Kleppmann, Martin (Oct 29, 2020). Distributed Systems. local page: Martin Kleppmann.
- ↑ Bacon, Jean (1993). Concurrent Systems: An Integrated Approach to Operating Systems, Database, and Distributed Systems (2nd ed.). local page: Addison Wesley. ISBN 978-0201177671.
- ↑ Kleppmann, Martin (Mar 1, 2017). Designing Data-Intensive Applications:The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. local page: O'reily Press. , Chapter 10: Batch Processing, Page 429
Related Pages
Author:Martin Kleppmann