Changeset [c9078230d44658f48ca4aed9a33ad84b00ad9581] by Shayan Pooya
May 26th, 2014 @ 11:33 AM
pipeline_utils: add get_grouping_lists function.
This function receives the current outputs of the previous stage
and the outputs
of the newly finished task. Based on these inputs, it produces two
lists.
1. The list of new output groups which should be consumed by a new
task.
2. The list of modified ouputs groups which should be transmitted
to the worker
task for consumption.
https://github.com/discoproject/disco/commit/c9078230d44658f48ca4ae...
Committed by Shayan Pooya
- M master/src/pipeline_utils.erl
- M master/test/pipeline_utils_test.erl
Create new ticket
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป
Disco is an open-source implementation of the Map-Reduce framework for distributed computing. As the original framework, Disco supports parallel computations over large data sets on unreliable cluster of computers.