The list should be checked for duplicate addresses too

Some people on internet suggest that looking like a Tails user you can still keep a reasonable anonymity set but I disagree

At the end of section "No filters":

Tails' decision shows their focus on privacy rather than real anonymity

This is only helpful for non-anonymous browsers. It harms anonymity if you intend to include it with Tor Browser

To help with the 4th criteria,

I think we can ignore the results when three of them are not very close. It should still be more efficient than multiple instances.

With the recent work on the time sources the results would be very close to NTP accuracy and setting offsets to no more than 1 or 2 seconds, I think it is not such a bad idea to have just a single sdwdate

What about having a single sdwdate for all Qubes which will add/substract some random skew for each VM?

maybe curl and grep the addresses in comments for a quick scan of onion address?

If you have time could you check how long it takes with 5 or 6 threads? I think it will be near equal to 8, not for reproducibility reasons just for efficient use of system resources. There is probably no reason to use 16 cores on a machine that supports it which would be overkill

Sorry for all this confusion, I think it is only a difference between whether the program "tries" to operate in a single-threaded mode or multi-threaded mode, when we use --threads=1 or don't specify it (default is 1) it compresses the whole file in a single block, however setting --threads 0 or bigger than 1 triggers the multi-threaded mode and the file is split into blocks depending on the compression level and then compressed resulting in a difference in the archive file. how many threads actually used is irrelevant. changing compression level or manually specifying the block sizes will change the outcome.

I will try this with xz utils binaries first in Windows host and then with half of the available cores in Windows VM

I may be wrong, the best way to test this is to maybe create the same archive with half of the available cores in a VM however I can't do this, I don't have debian stretch

I mean reproducibility "between" computers, not on the same one

Did you compare your --threads=30 archive with --threads=8 archive?

This quoted part indicates physical cpu threads:

Could you please check how long it takes with 4 threads, using %50 of cpu is expected, it does not necessarily mean it will take twice as long

I think in the worst case you could care less about a perfectly reproducible end archive (tar.xz) and instead focus on the extracted (tar) file being reproducible, for example linux kernel files are compressed either xz or gz but only the tar itself is signed

If you have 8 threads and if using more than 8 produces same checksum as 8, then what I said would be true

I think the default settings are optimal

But I have a feeling it would produce different archives with different number of threads, single core vs dual core vs quad core vs custom vm cores

awesome!

It seems you could use something like this with xz utils:

There is another tool called pxz (parallel xz): https://packages.debian.org/stretch/main/pxz

you could also try lowering or increasing the compression dictionary size to see how it affects the size and speed, however I don't know the commands

there is some related information here:
http://stackoverflow.com/questions/12313242/utilizing-multi-core-for-targzip-bzip-compression-decompression

xz doesn't seem to store file names or timestamps so it should be reproducible, you could still use tar with reproducible options to tar the files and maybe combine with p7zip's xz compression

Other than lowering the compression level, maybe tar doesn't support multi threaded compression whereas on 7zip with xz or 7z compression I can utilize all of my cpu cores.

any improvement?

It was just to remind that this "feature" is just the tip of an iceberg that keeps getting bigger over time. If you see it feasible and preferable to move away from systemd I would expect you to start the ticket/discussion

relevant:

Works for me, hid my cpu name

is there any improvement? how long did it take before?

@Patrick good news

Perhaps Patrick haven't tried bsdtar yet or perhaps it didn't work at the time he asked this question because the required version of bsdtar or other requirements was not in stable repositories

http://www.systutorials.com/qa/4/how-to-efficiently-archive-a-very-large-sparse-file
https://stackoverflow.com/questions/13252682/copying-a-1tb-sparse-file

also consider OSTIF

@Patrick try asking this on encode.ru. You may get the best answers there