Introduction Running data-intensive pipelines at scale can be complex, particularly when handling the distributed file storage and data…