Changeset [1aea4b253c650789a9345ce6babcb8b60ad1b620] by Shayan Pooya
June 30th, 2014 @ 10:01 AM
fair_scheduler: avoid comparing the list of the jobs with the size of the priority queue.
The ReduceJob in test_input was getting stuck in some scenarios.
The problem
was that MergeReduceJob finished sooner and was put in the NotJobs
list. The
priority queue only included the ReduceJob. Therefore, the length
of the NotJobs
was equal to the size of the priority list and schedule would have
returned
nojobs.
One workaround for this problem was to visit the the configure
page of disco
which resulted into calling the schedule() function again and the
job just
resumes.
In this commit we avoid comparing the size of the priority queue
and the notjobs
list and thus resolve this problem. Instead we only return nojobs
when we
exhaust the priority queue without finding a valid job.
https://github.com/discoproject/disco/commit/1aea4b253c650789a9345c...
Committed by Shayan Pooya
- M master/src/fair_scheduler_fair_policy.erl
Create new ticket
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป
Disco is an open-source implementation of the Map-Reduce framework for distributed computing. As the original framework, Disco supports parallel computations over large data sets on unreliable cluster of computers.