multithreading - Spring Batch thread-safe ItemReader (process indicator pattern) -

i'm implemented remote chunking using amqp (rabbitmq). need run parallel jobs within web container.

my simple controller (testjob use remote chunking):

@controller public class jobcontroller {      @autowired     private joblauncher joblauncher;      @autowired     private job testjob;      @requestmapping("/job/test")     public void test() {         jobparametersbuilder jobparametersbuilder = new jobparametersbuilder();         jobparametersbuilder.adddate("date",new date());         try {             joblauncher.run(personjob,jobparametersbuilder.tojobparameters());         } catch (jobexecutionalreadyrunningexception | jobrestartexception | jobparametersinvalidexception | jobinstancealreadycompleteexception e) {             e.printstacktrace();         }      }  } 

testjob reads data filesystem (master chunk) , send remote chunk (slave chunk). problem itemreader not thread safe.

there practical limitations of using multi-threaded steps common batch use cases. many participants in step (e.g. readers , writers) stateful, , if state not segregated thread, components not usable in multi-threaded step. in particular of off-the-shelf readers , writers spring batch not designed multi-threaded use. is, however, possible work stateless or thread safe readers , writers, , there sample (paralleljob) in spring batch samples show use of process indicator (see section 6.12, “preventing state persistence”) keep track of items have been processed in database input table.

i'm considered on paralleljob sample on spring batch github repository https://github.com/spring-projects/spring-batch/blob/master/spring-batch-samples/src/main/java/org/springframework/batch/sample/common/stagingitemreader.java

i'm bit confused process indicator pattern. can find more detailed information pattern?

if you're concerned itemreader instance shared across job invocations, can declare itemreader step scope , you'll new instance per invocation remove threading concerns.

but answer direct question process indicator pattern i'm not sure documentation on is. there sample of it's implementation in spring batch samples (the parallel job uses it).

the idea behind provide status records going process. @ beginning of job/step mark records in process. records committed, mark them processed. removes need track state in reader since state in db (your query looks records marked in process).