java - Stopping a job programmatically inside the reducer code -


suppose detect in input key/value in reducer code, code should run reducer doesn't continue anymore, emitted records @ output written output file , job stops no more further reduce happening?

stopping job might not idea. if need it, 1 way create own exception class, perhaps extending either interruptedexception or ioexception, , throw exception whenever condition arises when want quit.

your exception class may follows:

class quitreducerexception extends interruptedexception {        //parameterless constructor       public quitreducerexception() {}        //constructor accepts message       public quitreducerexception(string message)       {          super(message);       } } 

and in reduce method, may use follows:

@override  protected void reduce(text key, iterable values, context context) throws ioexception,interruptedexception {       ...       if(<condition quit happen>){           throw new quitreducerexception("quitting reducer due specified reason");// may add details of reason quitting , available in job logs (in stderr)       }       ...   } 


ps: not ensure outputs emitted current reducer committed output files. other reducer wasn't finished not commit files. while reducers complete, have committed outputs.


Comments

Popular posts from this blog

sublimetext3 - what keyboard shortcut is to comment/uncomment for this script tag in sublime -

java - No use of nillable="0" in SOAP Webservice -

ubuntu - Laravel 5.2 quickstart guide gives Not Found Error -