Task Bench: A Parameterized Benchmark for Evaluating Parallel Runtime Performance.

We present Task Bench, a parameterized benchmarkdesigned to explore the performance of parallel and distributed programming systems under a variety of application scenarios. Task Bench lowers the barrier to benchmarkingmultiple programming systems by making the implementation for a given system orthogonal to the benchmarksthemselves: every benchmarkconstructed with Task Bench runs on every Task Bench implementation. Furthermore, Task Bench's parameterization enables a wide variety of benchmarkscenarios that distill the key characteristics of larger applications. We conduct a comprehensive study with implementations of Task Bench in 15 programming systems on up to 256 Haswell nodes of the Corisupercomputer. We introduce a novel metric, minimum effective task granularity to study the baseline runtime overhead of each system. We show that when running at scale, 100 {\mu}s is the smallest granularity that even the most efficient systems can reliably support with current technologies. We also study each system's scalability, ability to hide communication and mitigate load imbalance.