Why is my Spark job driver launched but not any of the tasks?

This is most likely because there is no available resource to launch the tasks, as if you run Spark with the default fine-grain mode, it requires at least 2 CPUs on a slave to launch. You only need 1 CPU if you launch with coarse-grain mode.

Have more questions? Submit a request

Comments

Powered by Zendesk