java - Why do I see multiple running instances of the same service (Elasticsearch) on Ubuntu, but only one is responsive? -
i running elasticsearch (es) on small aws ubuntu box, , working on tuning performance of box overall.
after recent deploy using saltstack, noticed number of running instances went 2 3 -- after being @ 2 several months. uptick in instances seems correspond uptick in memory usage.
i confirmed ps there 3 java processes running on box:
pid tty time cmd 9295 ? 00:02:08 java 14398 ? 00:00:12 java 26175 ? 00:40:48 java
when stop es command "sudo service elasticsearch stop", still left 2 es processes running according ps:
pid tty time cmd 9295 ? 00:02:08 java 26175 ? 00:40:48 java
i restarted service , had 3 again. seems strange 2 me because seemed 2 of services unresponsive stop command. (could so-called zombie or orphan process?)
i manually killed 3 processes , restarted es, , have single es instance. wondered if these wayward java processes related other service, after killing three, new relic confirmed large drop in memory usage , processes -- es-related processes:
my question why, after deploy, number of running instances go up? there functional elasticsearch reason this, or bug? cause either elasticsearch or service on ubuntu in general go unresponsive state?
any insight appreciated!
what when running command:
lsof -i :9200-9399 | tail -n +2 | awk '{print $2}' | xargs ps -p
lsof -i :9200-9399
list open files on port range 9200-9399, i.e. default port range used es. change range if configuration differs.
tail -n +2
ditch first output line lsof
command, contains column headers , it's not useful
awk '{print $2}'
fetch process id (pid) lsof
output
finally, xargs ps -p
run ps
command find out process runs under pid fetched awk
command.
you should output below , might started in investigation.
pid tty time cmd 21199 ttys011 4:39.26 /usr/bin/java -xms256m -xmx1g.... 22234 ttys012 5:12.22 /usr/bin/java -xms256m -xmx1g.... 23444 ttys013 3:33.54 /usr/bin/java -xms256m -xmx1g....
Comments
Post a Comment