c++ - Tensorflow thread pools appear to spin forever, even after session is closed -
the quick version:
i'm using tensorflow in ios project , seeing high cpu usage after close tensorflow session so:
auto status = session->close(); delete session; session = nil;
details:
having read code, i'm working under assumption under covers tensorflow using directsession
subclass of session
, explicitly cleans thread pools if it's configured own own threads. ensure this, updated config when creating session:
tensorflow::sessionoptions options; options.config.clear_session_inter_op_thread_pool(); options.config.set_use_per_session_threads(true); auto status = tensorflow::newsession(options, &session);
but still in profiler see cpu usage in eigen::nonblockingthreadpoolimpl<tensorflow::thread::eigenenvironment>::workerloop
continue increase indefinitely after closing session. seems steal
loop never stops.
if force down single thread there no steal
ing. so:
options.config.set_inter_op_parallelism_threads(1);
then tensorflow stops hogging cpu, threads still exist - they're suspended, waiting (presumably) woken process new tasks. isn't real solution problem - want able use many threads optimal.
Comments
Post a Comment