Page MenuHomePhabricator

Proposed Download Directory Structure / download redirects / stable download links / permalinks
Closed, ResolvedPublic

Description

the debian team uses a directory structure for their iso files that uses a "current" directory where the latest version of the installation disks can always be found. this allows for various scripting/bash commands from a user to always be able to download the latest version of a particular release, regardless of the version point release number.

for example the following command will always download the latest version of the amd64 debian netinst installer for stretch with the proprietary drivers:

wget -c -e robots=off -r -l1 -nd -P ./ -np "https://cdimage.debian.org\
/cdimage/unofficial/non-free/cd-including-firmware/current/amd64/iso-cd" \
-A "firmware-9.*.*-amd64-netinst.iso"

this helps greatly with documentation efforts, since documentation does not break and need updating based on a new point release being available.

whonix, however, currently uses a directory structure that is specific to point release names, which are linked from the webpage. would it be problematic to implement a "current" directory in a manner similar to debian's that hosts the latest whonix point release?

https://forums.whonix.org/t/permalink-download-links/8190
https://www.whonix.org/wiki/Dev/Redistribution#Update_Permanent_Links

Details

Impact
Low

Event Timeline

tempest triaged this task as Wishlist priority.Feb 24 2019, 1:00 PM
tempest created this task.

I requested such a feature myself once from Tor Project. They refused this because they said, when you ask users what version they are using, they will reply latest.

Not sure this is a good enough reason to refuse this?

it seems a rather odd reason to me since debian has literally done it this way for years. i understand that it would be a problem if the downloaded file name was always the same (for example, "tor-current.exe"). but, since the filenames always include the version number, there are multiple means of documenting for a user how to determine their version number before making support requests.

Alright. If you know how to automate / implement that... Using mediawiki or else... Would speed up implementation a ton.

I don't have ideas how to implement this. Patches are Welcome.

It doesn't strike me as too hard to just add a 'current' symlink pointing to the latest release?

What is the current process of uploading new images/releases onto the server? Can we not just add a step to adjust that symlink, at the end of the process?

On the other hand, I'm not sure how much time this really solves. Current links to downloads in the wiki look like this: https://download.whonix.org/ova/14.0.1.4.4/Whonix-XFCE-14.0.1.4.4.ova

Whilst the above wget command does use a wildcard approach to avoid explicitly stating *some* of the version name, you would have to change the download instructions to use such a method, no? Otherwise you *will* still have to list the full filename, with its version contained therein, as something like https://download.whonix.org/ova/current/Whonix-XFCE-14.0.1.4.4.ova, and so you haven't saved much time. Do you want to really replace hyperlinks with a code block showing a wget command like the above?

Another approach might be to use Nginx redirects (and a shell script or something, to maintain changes as new versions come out), so that URLs like https://download.whonix.org/ova/current/Whonix-XFCE-current.ova redirect to https://download.whonix.org/ova/14.0.1.4.4/Whonix-XFCE-14.0.1.4.4.ova . Useful?

Hypothetical:

/usr/local/bin/update-redirects.sh --new-version=14.0.1.4.4
# (automatically updates redirects file to use the new version and reloads nginx)
In T895#18241, @mig5 wrote:

It doesn't strike me as too hard to just add a 'current' symlink pointing to the latest release?

I've tried a symlink.

mkdir /var/rsync/ova/current

ln -s /var/rsync/ova/14.0.1.4.4/Whonix-XFCE-14.0.1.4.4.ova /var/rsync/ova/current/Whonix-XFCE-current.ova

Problem: it changes the filename to Whonix-XFCE-current.ova. Fixable?

What is the current process of uploading new images/releases onto the server?

rsync ( https://github.com/Whonix/whonix-developer-meta-files/blob/master/release/upload_images )

Can we not just add a step to adjust that symlink, at the end of the process?

upload_images indeed could have a switch for stable vs testers-only (different or no symlink).

On the other hand, I'm not sure how much time this really solves. Current links to downloads in the wiki look like this: https://download.whonix.org/ova/14.0.1.4.4/Whonix-XFCE-14.0.1.4.4.ova

Whilst the above wget command does use a wildcard approach to avoid explicitly stating *some* of the version name, you would have to change the download instructions to use such a method, no? Otherwise you *will* still have to list the full filename, with its version contained therein, as something like https://download.whonix.org/ova/current/Whonix-XFCE-14.0.1.4.4.ova, and so you haven't saved much time. Do you want to really replace hyperlinks with a code block showing a wget command like the above?

I am indeed not a fan of a long complicated download command. (It's a user contributed guide on its own domain name.)

@Patrick:

Added this to /etc/nginx/sites-enabled/download.whonix.org.conf:

include /etc/nginx/conf.d/download_redirects;

The contents of that file, just 3 lines:

set $version "14.0.1.4.4";
rewrite ^/ova/current(?:/)?$ /ova/$version permanent;
rewrite ^/ova/current/Whonix-(.*)-current.(.*)$ /ova/$version/Whonix-$1-$version.$2 permanent;

Now you can simply hit https://download.whonix.org/ova/current/Whonix-CLI-current.ova and it will redirect with a 301 so that the filename ends up as https://download.whonix.org/ova/14.0.1.4.4/Whonix-CLI-14.0.1.4.4.ova

So all your links in the webpages could be to the non-version-specific URL.

Users could also browse to https://download.whonix.org/ova/current/ and they'll be redirected to https://download.whonix.org/ova/14.0.1.4.4

No symlinks required.

Then your upload script only needs to do a sed on the /etc/nginx/conf.d/download_redirects to change the $version variable from 14.0.1.4.4 to something else, e.g to change to 15.0.0.0.7:

sed -i 's/version ".*";$/version "15.0.0.0.7";/' /etc/nginx/conf.d/download_redirects
service nginx reload

Happy with that? We would keep that file download_redirects outside of puppet so that it doesn't clobber any changes made by you or your script.

Edited above comment a few times to fix syntax

I suggest not permanent redirection, otherwise browsers may cache old version.

@Patrick I have set it to a temporary redirect now (302). In my tests in Firefox, the request is not being cached (server sends back the 302 each time according to Nginx logs)

I tried fiddling with adding no-cache headers too but it doesn't seem necessary (and I couldn't get it to work without spending more time)

Patrick renamed this task from Proposed Download Directory Structure to Proposed Download Directory Structure / download redirects / stable download links.May 3 2019, 10:08 AM
Patrick renamed this task from Proposed Download Directory Structure / download redirects / stable download links to Proposed Download Directory Structure / download redirects / stable download links / permalinks.May 3 2019, 10:13 AM

Awesome!

Does this work for you? @tempest

@Patrick @mig5 it appears to be working great. thank you and sorry for the late reply.

Patrick updated the task description. (Show Details)