rust - Is there any way to implement the Send trait for ZipFile? -


i want read .zip file in different thread using zip crate.

extern crate zip;  use zip::ziparchive; use zip::read::zipfile; use std::fs::file; use std::io::bufreader; use std::thread;  fn compute_hashes(mut file: zipfile) {     let reader_thread= thread::spawn(move || {         let mut reader = bufreader::new(file);         /* ... */     }); }  fn main() {     let mut file = file::open(r"facebook-jakubonderka.zip").unwrap();     let mut zip = ziparchive::new(file).unwrap();      in 0..zip.len() {         let mut inside = zip.by_index(i).unwrap();          if !inside.name().ends_with("/") { // skip directories             println!("filename: {}", inside.name());             compute_hashes(inside);         }     } }

but compiler shows me error:

error[e0277]: trait bound `std::io::read: std::marker::send` not satisfied   --> src/main.rs:10:24    | 10 |     let reader_thread= thread::spawn(move || {    |                        ^^^^^^^^^^^^^ `std::io::read` cannot sent between threads safely    |    = help: trait `std::marker::send` not implemented `std::io::read`    = note: required because of requirements on impl of `std::marker::send` `&mut std::io::read`    = note: required because appears within type `std::io::take<&mut std::io::read>`    = note: required because appears within type `zip::crc32::crc32reader<std::io::take<&mut std::io::read>>`    = note: required because appears within type `zip::read::zipfilereader<'_>`    = note: required because appears within type `zip::read::zipfile<'_>`    = note: required because appears within type `[closure@src/main.rs:10:38: 13:6 file:zip::read::zipfile<'_>]`    = note: required `std::thread::spawn`

but same works type std::fs::file. necessary fix zip crate or there other method?

this a limitation of zip crate's api , can't change anything. problem file ziparchive created calling new , passing reader -- implements read , seek. these requirements reader (in particular, reader doesn't need clone). thus, whole ziparchive can own 1 reader.

but ziparchive able produce zipfiles implement read themselves. how work if whole ziparchive has 1 reader? through sharing! reader shared between archive , files. sharing not thread save! mutable reference reader stored in each zipfile -- violates rust's core principle.

this known issue of crate , being discussed on github issue tracker.

so can now? not whole lot, few possibilities (as mentioned library author) might ok use case:

  • you decompress whole file memory first, send raw data thread calculations on it. like:

    let data = vec::new(); bufreader::new(file).read_to_end(&mut data)?; let reader_thread= thread::spawn(move || {     // stuff `data` });

    but if want compute cheap hash function on files, loading contents memory slower computing hash on fly , might infeasible if files big.

  • creating 1 ziparchive each thread. might slow if have many small files in archive...

a tiny hint: starting thread costs time. don't want start thread each unit of work, rather maintain fixed number of threads in thread pool, manage work in queue , assign work idle worker threads. the threadpool crate might serve needs.


Comments

Post a Comment

Popular posts from this blog

PHP and MySQL WP -

i trying accomplish 2 things here code. writing php directly wp page using insert php plugin. 1) query mysql database , return results 1 column 4 columns. 2) make each result own hyperlink directs user new wp page runs new query on page. i able query show results , turn results dynamic hyperlink, cannot formatting down. right returns result 1 column. here have: [insert_php] $servername = "localhost"; $username = "login"; //edited privacy $password = "password"; //edited privacy $database = "database"; //edited privacy // create connection $conn = new mysqli($servername, $username, $password, $database); // check connection if ($conn->connect_error) { die("connection failed: " . $conn->connect_error); } echo "connected successfully"; //get results database $result = mysqli_query($conn,"select mine mines order mine"); $all_property = array(); //declare array saving property //showing proper...
Read more

android - InAppBilling registering BroadcastReceiver in AndroidManifest -

i'm implementing inapp billing in android app. works well, however, i'm trying decouple broadcast receiver activity manifest. suggestion in android's trivialdrive sample: // important: dynamically register broadcast messages updated purchases. // register receiver here instead of <receiver> in manifest // because call getpurchases() @ startup, therefore can ignore // broadcasts sent while app isn't running. // note: registering listener in activity bad idea, done here // because sample. regardless, receiver must registered after // iabhelper setup, before first call getpurchases(). currently there's class extends broadcastreceiver : public class iabbroadcastreceiver extends broadcastreceiver { /** * intent action receiver should filter for. */ public static final string action = "com.android.vending.billing.purchases_updated"; private final iabbroadcastlistener mlistener; public iabbroadcastreceiver(iabbroadcastlistener listener) { m...
Read more

go - golang pprof for c library code -

i have been trying profile golang application pprof, seems details of c code(linked static library) missed in output result. flame graph from frame graph can see c code time consuming reported within "runtime._externalcode", no details @ all. , sure static library compiled debug enable. so there way make pprof details(like stack trace) of 3rd party c code? or there other golang profiling tool this?
Read more