java - Why do I see multiple running instances of the same service (Elasticsearch) on Ubuntu, but only one is responsive? -


i running elasticsearch (es) on small aws ubuntu box, , working on tuning performance of box overall.

after recent deploy using saltstack, noticed number of running instances went 2 3 -- after being @ 2 several months. uptick in instances seems correspond uptick in memory usage. enter image description here

i confirmed ps there 3 java processes running on box:

pid tty          time cmd 9295 ?         00:02:08 java 14398 ?        00:00:12 java 26175 ?        00:40:48 java 

when stop es command "sudo service elasticsearch stop", still left 2 es processes running according ps:

pid tty          time cmd 9295 ?         00:02:08 java 26175 ?        00:40:48 java 

i restarted service , had 3 again. seems strange 2 me because seemed 2 of services unresponsive stop command. (could so-called zombie or orphan process?)

i manually killed 3 processes , restarted es, , have single es instance. wondered if these wayward java processes related other service, after killing three, new relic confirmed large drop in memory usage , processes -- es-related processes:

enter image description here

my question why, after deploy, number of running instances go up? there functional elasticsearch reason this, or bug? cause either elasticsearch or service on ubuntu in general go unresponsive state?

any insight appreciated!

what when running command:

 lsof -i :9200-9399 | tail -n +2 | awk '{print $2}' | xargs ps -p 

lsof -i :9200-9399 list open files on port range 9200-9399, i.e. default port range used es. change range if configuration differs.

tail -n +2 ditch first output line lsof command, contains column headers , it's not useful

awk '{print $2}' fetch process id (pid) lsof output

finally, xargs ps -p run ps command find out process runs under pid fetched awk command.

you should output below , might started in investigation.

  pid tty           time cmd 21199 ttys011    4:39.26 /usr/bin/java -xms256m -xmx1g.... 22234 ttys012    5:12.22 /usr/bin/java -xms256m -xmx1g.... 23444 ttys013    3:33.54 /usr/bin/java -xms256m -xmx1g.... 

Comments

Popular posts from this blog

python - pip install -U PySide error -

arrays - C++ error: a brace-enclosed initializer is not allowed here before ‘{’ token -

cytoscape.js - How to add nodes to Dagre layout with Cytoscape -