java - Stopping a job programmatically inside the reducer code -
suppose detect in input key/value in reducer code, code should run reducer doesn't continue anymore, emitted records @ output written output file , job stops no more further reduce happening?
stopping job might not idea. if need it, 1 way create own exception class, perhaps extending either interruptedexception
or ioexception
, , throw exception whenever condition arises when want quit.
your exception class may follows:
class quitreducerexception extends interruptedexception { //parameterless constructor public quitreducerexception() {} //constructor accepts message public quitreducerexception(string message) { super(message); } }
and in reduce method, may use follows:
@override protected void reduce(text key, iterable values, context context) throws ioexception,interruptedexception { ... if(<condition quit happen>){ throw new quitreducerexception("quitting reducer due specified reason");// may add details of reason quitting , available in job logs (in stderr) } ... }
ps: not ensure outputs emitted current reducer committed output files. other reducer wasn't finished not commit files. while reducers complete, have committed outputs.
Comments
Post a Comment