Spark REPL doesn't run due to an error -


i got pre-built spark 1.4.1 , i'm running hdp 2.6. when run spark-shell gives me error message follows. because i'm missing jar file ?

 error actor.oneforonestrategy: java.lang.nullpointerexception     @ org.apache.spark.deploy.client.appclient$clientactor$$anonfun$receivewithlogging$1.applyorelse(appclient.scala:160)     @ scala.runtime.abstractpartialfunction$mcvl$sp.apply$mcvl$sp(abstractpartialfunction.scala:33)     @ scala.runtime.abstractpartialfunction$mcvl$sp.apply(abstractpartialfunction.scala:33)     @ scala.runtime.abstractpartialfunction$mcvl$sp.apply(abstractpartialfunction.scala:25)     @ org.apache.spark.util.actorlogreceive$$anon$1.apply(actorlogreceive.scala:59)     @ org.apache.spark.util.actorlogreceive$$anon$1.apply(actorlogreceive.scala:42)     @ scala.partialfunction$class.applyorelse(partialfunction.scala:118)     @ org.apache.spark.util.actorlogreceive$$anon$1.applyorelse(actorlogreceive.scala:42)     @ akka.actor.actor$class.aroundreceive(actor.scala:465)     @ org.apache.spark.deploy.client.appclient$clientactor.aroundreceive(appclient.scala:61)     @ akka.actor.actorcell.receivemessage(actorcell.scala:516)     @ akka.actor.actorcell.invoke(actorcell.scala:487)     @ akka.dispatch.mailbox.processmailbox(mailbox.scala:238)     @ akka.dispatch.mailbox.run(mailbox.scala:220)     @ akka.dispatch.forkjoinexecutorconfigurator$akkaforkjointask.exec(abstractdispatcher.scala:393)     @ scala.concurrent.forkjoin.forkjointask.doexec(forkjointask.java:260) 

what problem for?

i got version of scala download , diferent error when running scala-shell. here error.

 warn remoting: tried associate unreachable remote address [akka.tcp://sparkmaster@headnodehost:7077]. address gated 5000 ms, messages address delivered dead letters. reason: headnodehost 15/07/23 15:25:59 error cluster.sparkdeployschedulerbackend: application has been killed. reason: masters unresponsive! giving up. 15/07/23 15:25:59 warn cluster.sparkdeployschedulerbackend: application id not initialized yet. 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/stages/stage/kill,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/api,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/static,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/executors/threaddump/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/executors/threaddump,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/executors/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/executors,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/environment/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/environment,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/storage/rdd/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/storage/rdd,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/storage/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/storage,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/stages/pool/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/stages/pool,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/stages/stage/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/stages/stage,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/stages/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/stages,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/jobs/job/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/jobs/job,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/jobs/json,null} 15/07/23 15:25:59 info handler.contexthandler: stopped o.s.j.s.servletcontexthandler{/jobs,null} 15/07/23 15:25:59 info ui.sparkui: stopped spark web ui @ http://192.168.164.136:4040 15/07/23 15:25:59 info scheduler.dagscheduler: stopping dagscheduler 15/07/23 15:25:59 info cluster.sparkdeployschedulerbackend: shutting down executors 15/07/23 15:25:59 info cluster.sparkdeployschedulerbackend: asking each executor shut down 15/07/23 15:25:59 error actor.oneforonestrategy: java.lang.nullpointerexception     @ org.apache.spark.deploy.client.appclient$clientactor$$anonfun$receivewithlogging$1.applyorelse(appclient.scala:160)     @ scala.runtime.abstractpartialfunction$mcvl$sp.apply$mcvl$sp(abstractpartialfunction.scala:33)     @ scala.runtime.abstractpartialfunction$mcvl$sp.apply(abstractpartialfunction.scala:33)     @ scala.runtime.abstractpartialfunction$mcvl$sp.apply(abstractpartialfunction.scala:25)     @ org.apache.spark.util.actorlogreceive$$anon$1.apply(actorlogreceive.scala:59)     @ org.apache.spark.util.actorlogreceive$$anon$1.apply(actorlogreceive.scala:42)     @ scala.partialfunction$class.applyorelse(partialfunction.scala:118) 

based on comments , other recent questions, problem you've downloaded "pre-build user provided hadoop" version of apache spark. instead should download build hadoop version have: 2.6 in case.

the download link apache spark 1.4.1 built hadoop 2.6 http://d3kbcqa49mib13.cloudfront.net/spark-1.4.1-bin-hadoop2.6.tgz


Comments

Popular posts from this blog

python - pip install -U PySide error -

arrays - C++ error: a brace-enclosed initializer is not allowed here before ‘{’ token -

cytoscape.js - How to add nodes to Dagre layout with Cytoscape -