Page MenuHomePhabricator

Qubes templates: graphical updater (Apper) broken
Closed, ResolvedPublic

Description

Issue:

The graphical updater (Apper) broken is broken. It reports no upgrades, even if there are upgrades.

prerequsite knowledge:

TODO decision:

TODO technical:

  • remove apper?
  • remove apper menu entry?

Details

Impact
Normal

Event Timeline

Patrick created this task.Jul 4 2015, 1:39 AM
Patrick updated the task description. (Show Details)
Patrick raised the priority of this task from to Normal.
Patrick set Impact to Normal.

I have not tested it lately, but the same goes for the regular Debian templates ... sort of.

I wait about 6 minutes after starting the TemplateVM and try the update, it works. There is a timer that goes off 5 minutes after template is started that does an apt-get update. When the template is first started, that has not happened yet.

May be worth while to see if Apper can do an update first, or add a wrapper to do it.

Patrick updated the task description. (Show Details)Jul 4 2015, 4:41 AM

(Messed up ticket description. Fixed now.)

On https://www.whonix.org/wiki/Dev/Automatic_Updates I'll explain, why graphical updaters are insecure, a usability mess and why replacements have a scope of maintain a complete separate upstream project. I think in the absence of secure and usable gui utilities, we should avoid them and refer to cli tools. That's awful, but the only workable secure choice I can see at this time.

An Apper security issue...

Functional sources.list snippet.

deb http://security.debian.org wheezy/updates main contrib non-free

Non-functional sources.list snippet with a purposed typo for demonstration purposes.

deb http://ecurity.debian.org wheezy/updates main contrib non-free

Apper won't complain about this. Therefore with a trivial DNS based denial of service, a user can be prevented to fetch security updates. Therefore in my opinion Apper disqualifies for an update gui for a security focused operating system.

Or what about writing some utility, maybe even part of a wrapper that verifies the source.list sources first?

Apper will notify you if a package is not signed.

maybe even part of a wrapper that verifies the source.list sources first?

Btw, it's not just about messed up sources.list. This was only a cheap trick to demonstrate the issue. The same would happen if an access level (wifi) or ISP level adversary would run an active man-in-the-middle attack preventing the connectoin to the security repository.

I guess a simple sanity test, checking if the right security repository is enabled in sources.list would be a good feature for whonixcheck or some other maybe-existing more general security check tool for Debian.

The code for finding typos that lead to DNS or connection errors might be already written. (packagekit)

Or what about writing some utility,

It's a huge task of the size of a separate project.

Tried to explain why this is difficult to invent a secure gui based apt-get upgrader in this very chapter: https://www.whonix.org/wiki/Dev/Automatic_Updates#One_click_upgrade_button_for_Debian_packages.

Few more keywords to illustrate why this is a complex project... It follows a list of stuff from the top of my head that is difficult to handle in a gui upgrader

  • debconf questions
  • dpkg interactive conflict resolution dialog
  • DNS failures
  • connection failures (ISP's preventing connections to apt repositories for whatever reason; targeted attacks)
  • installation issues due to broken dependencies
  • hold back packages
  • notifications about packages available for autoremoval
  • packages in Debian stable, that that have no security support (T135)
  • slow retrival attacks (Defined by TUF: Attacks and Weaknesses (w).)
  • endless data attacks (Defined as per TUF.)
  • retrieved package release who's valid-until field [when forgotten to refresh the repository, eventual roll back (TUF) or indefinite freeze (TUF) attack]
  • invalid signatures
  • expired signing keys
  • interrupted upgrades
  • failing Debian maintainer scripts
  • locked apt (some other process still using apt or died)
  • permission issues

Instead of inventing yet another gui upgrader - which is a giant task to do securely - I think that time should be rather used to work on the fixing the existing ones. Such as apper or (ubuntu-)software-center. (software-center was ported to Debian wheezy, but didn't make into Debian jessie.)

Updating using a gui is really pretty. Works in many cases. But in the corner cases, you have to refer users back to console based apt-get upgrades. Since secure upgrades are currently not possible using gui tools, because the existing ones don't handle [at least inform] many failures, I think the reasonable thing for security focused Debian derivatives is to train users to use apt-get command line. They get to know how it's normally looking and are more likely to notice any issues.

However, what could be done and with reasonable work and maintenance effort would be a secure gui update notification tool. Using a /usr/lib/apt-get-wrapper like approach. Just notification if updates are available or not. Like whonixcheck does. No actual apt-get upgrade.

I agree inventing a new gui updater is not ideal.

Out of the available ones, which handles the most keyword items? Anything lacking we can write a wrapper or make changes, patches and push upstream.

One thing we are tasked with is making it easier for end user to use. There are different use cases for different types of users, but I believe requiring a user to user any command line is not friendly at all, but may be required for certain use case where security is essential.

So is the a GUI that fits most of our needs; especially one like Apper that is already available.

Agreed, cli usage is user unfriendly. Crystal clear. No argument from me. One of the main reasons why mainstream users are scared of Linux distributions.


I can do my best testing for keyword items, reporting bugs upstream. (Once we picked a gui upgrader.) Could you attach patches to those in python or C++? (Depending on which one we choose?)


Choosing one is not simple. It's not just about keyword items.

Let's brainstorm more criteria for choosing which one deserves our contributions...

  • keyword items
  • language written in for simplicity of patches: I guess you prefer python over C and C++?
  • language written in for security: python preferred over C
  • future proof: Is the project going to be abandoned anytime soon?
  • usability of the tool
  • responsiveness upstream: How cooperative upstream is, how well you get along with them.
  • activity upstream: If they're still alive, active and kicking or if the project is barely left alive.
  • source code management used by upstream: git or bazzar or worse?
  • dependencies: okay to depend on KDE or so?
  • packages already in Debian [/ Fedora] repository: Work saving, if already packaged there, so the packaging work and getting it upstream is already done.

A few I have in mind:

ubuntu-software-center

  • Good: usability, Last time I checked, it was the most pretty one. Best usability - apart from it's bugs such as crashes. Maybe they fixed that.
  • Good: written in python
  • Bad: Has "ubuntu" in it's name. Doesn't demonstrate their interest in having it work well in Debian.
  • Bad: (imho) using bazzar. [Perhaps I am too lazy, ignorant and wouldn't know why they bother with something other than git.]
  • Bad: Not in Debian repository. (Removed since jessie.)
  • Question: It's also political. Would you like to try to talk ubuntu-software-center developers about becoming agnostic, fully supporting debian?

Synaptic:

  • Bad: Old. Usability sux imho. Not much progress over the years.

Apper:

  • Bad: dependencies: depends on KDE
  • Bad: written in C++
  • Medium: usability, oh well...
  • Good: Using packagekit by freedesktop.org, the ones who care about standards and doing things the-right-way(tm)

gnome-packagekit:

  • Bad: written in C
  • usability: worse than Apper?

Looks like there are not that many to choose from.

Patrick added a subscriber: bnvk.Jul 7 2015, 5:48 PM
mfc added a comment.Jul 8 2015, 4:27 AM

Don't forget Muon, which is Qt-based, at least Kubuntu uses it. Could be
worth evaluating.

List of more:
https://en.wikipedia.org/wiki/Advanced_Packaging_Tool#Front-ends

Patrick (Patrick Schleizer):

Patrick added a comment.

Agreed, cli usage is user unfriendly. Crystal clear. No argument from me. One of the main reasons why mainstream users are scared of Linux distributions.


I can do my best testing for keyword items, reporting bugs upstream. (Once we picked a gui upgrader.) Could you attach patches to those in python or C++? (Depending on which one we choose?)


Choosing one is not simple. It's not just about keyword items.

Let's brainstorm more criteria for choosing which one deserves our contributions...

  • keyword items
  • language written in for simplicity of patches: I guess you prefer python over C and C++?
  • language written in for security: python preferred over C
  • future proof: Is the project going to be abandoned anytime soon?
  • usability of the tool
  • responsiveness upstream: How cooperative upstream is, how well you get along with them.
  • activity upstream: If they're still alive, active and kicking or if the project is barely left alive.
  • source code management used by upstream: git or bazzar or worse?
  • dependencies: okay to depend on KDE or so?
  • packages already in Debian [/ Fedora] repository: Work saving, if already packaged there, so the packaging work and getting it upstream is already done.

    -----

    A few I have in mind:

    ubuntu-software-center
  • Good: usability, Last time I checked, it was the most pretty one. Best usability - apart from it's bugs such as crashes. Maybe they fixed that.
  • Good: written in python
  • Bad: Has "ubuntu" in it's name. Doesn't demonstrate their interest in having it work well in Debian.
  • Bad: (imho) using bazzar. [Perhaps I am too lazy, ignorant and wouldn't know why they bother with something other than git.]
  • Bad: Not in Debian repository. (Removed since jessie.)
  • Question: It's also political. Would you like to try to talk ubuntu-software-center developers about becoming agnostic, fully supporting debian?

    Synaptic:
  • Bad: Old. Usability sux imho. Not much progress over the years.

    Apper:
  • Bad: dependencies: depends on KDE
  • Bad: written in C++
  • Medium: usability, oh well...
  • Good: Using packagekit by freedesktop.org, the ones who care about standards and doing things the-right-way(tm)

    gnome-packagekit:
  • Bad: written in C
  • usability: worse than Apper?

    Looks like there are not that many to choose from.

TASK DETAIL

https://phabricator.whonix.org/T373

EMAIL PREFERENCES

https://phabricator.whonix.org/settings/panel/emailpreferences/

To: Patrick
Cc: HulaHoop, Patrick, nrgaway, marmarek, mfc, MemoryLost, WhonixQubes, troubadour

bnvk added a comment.Jul 9 2015, 5:13 PM

I definitely agree that the faster a FOSS OS can get to a GUI
installer / updater, the fast it will gain user adoption. CLI 's
are rife with so many challenges for a user.

I don't know enough about these options as I'm still a recovering
Mac user (just started with Qubes OS), but the Ubuntu one
*sounds* the best, except for the coupled to Ubuntu challenge :-)

Great criteria. Big+ for opting for Python instead of C!

From a usability point of view, ubuntu-software-center is the most, if not the only, ambitious and serious one.

I think this should become a proper, generic, distribution agnostic [maybe Debian only] upstream project. It is too important and difficult security wise to have this as a quick and dirty Qubes-only patchset. By having it available in Debian, therefore by a lot derivative distributions, much more people would look into this.

Are you up for probably maintaining software-center in Debian? @nrgaway? Or is this too much of a daunting, mammoth task? Or would we require a dedicated developer for this?

I will have to take a look at the current source of software-center and play around with it a bit to be able to provide that answer. I will look into it this weekend :)

Is software-center subject to Canonical's CLA? The political problems with their licensing is one part. The other is most of their solutions are usually a product of NIH syndrome, are not designed with any other distro in mind but Ubuntu and they end up as abandonware while Canonical moves onto their next thing.

For long-term you might want to invest your effort in an upstream that has a track record of dedication for their software.

In T373#5894, @nrgaway wrote:

I will have to take a look at the current source of software-center and play around with it a bit to be able to provide that answer. I will look into it this weekend :)

What's your conclusions?

I did have a look at the software over the weekend, which BTW is GNU GPL v3 licensed.

I don't think it would be too much of a task to maintain software-center in Debian but I have concerns about supporting software-center being:

  • I am unsure if this would solve the GUI update requirement. Now, I do not have Ubuntu installed to test, but isn't the software-center package just used to search and install packages? From what I recall, Ubuntu still used packagekit or something similar to handle the updating process of packages.
  • As previously mentioned, it does not support Fedora or RPM package backend
  • A RPM backend could be added, but would take time to implement
  • there are many features of software-center we do not need (or do we) such as reviews and app purchases. It adds an extra layer of complexity to configure for these features.
  • software-center is about 60,000 lines of code. There are other solutions like lubuntu-software-center that may be worth considering as well.

I think what makes most sense is to either report the security issues or fix packagekit's aptcc backend for apt and submit upstream?

  • As previously mentioned, it does not support Fedora or RPM package backend
  • A RPM backend could be added, but would take time to implement

In long term, not required. The plan is to throw out Fedora anyhow. Source:
https://github.com/QubesOS/qubes-issues/issues/1054#issuecomment-119448165

  • there are many features of software-center we do not need (or do we) such as reviews and app purchases. It adds an extra layer of complexity to configure for these features.

Agreed. So this would require an if/else patch to have this code disabled on non-Ubuntu?

There are other solutions like lubuntu-software-center that may be worth considering as well.

Yes.

I think what makes most sense is to either report the security issues or fix packagekit's aptcc backend for apt and submit upstream?

Contributing to packagekit seems sensible in any case. Since most new graphical package managers seem to be based on packagekit nowadays. WIth respect to noninteractivness and no questions during installation (Hughsie's Law) they seem to have the right mindset.

https://www.phoronix.com/scan.php?page=news_item&px=Ubuntu-MATE-Drops-USC

This Ubuntu "Software Store" has already been removed from the daily images of this MATE desktop spin of Ubuntu, ahead of the 15.10 Alpha 2. Wimpress mentioned they have "something else lined up by way of replacement" but that it's not the Debian Synaptic UI.

Martin Wimpress shared these details via this Google+ post. This decision only affects the Ubuntu MATE version and not other members of the Ubuntu family. The Ubuntu Software Center has been criticized recently for being bloated, not well maintained, and in need of a full rewrite.

Patrick updated the task description. (Show Details)

I am afraid to say, but apparently this one stalled without any long term plan on how to proceed.

I am still working on this. I spent 3 days on it a few weeks ago. I ended up reviewing the source code of many packages, and did a short audit on packagekit and gnome-packagekit to find ways to resolve issue.

Step one was to get gpk working in both Fedora and Debian. It works in Debian, not so much in fc21 (well, it did not a few weeks ago).

Next step was to get it to update dpkg index on start. From what I can remember from memory and without reviewing my notes, the newer version of packagekit removed portions of the update code and relies on the unattended-upgrades package for periodic updates.

I understand though that there is a way to force the index to update still in Debian when gpk is launched from gnome's settings menu; which I have not had success in locating yet. I am going to install a HVM version of standard Debian and test there.

Once these steps are complete, we can apply any security patches to packagekit backend aptcc? (again from memory) and submit upstream.

Timeline is October since I have other deadlines items to complete first but plan to work on this as often as time permits.

nrgaway claimed this task.Aug 19 2015, 9:21 PM
mfc added a comment.Sep 21 2015, 2:18 PM

Honestly I think this issue has ballooned into a bunch of different (but related) issues.

The "prerequisite knowledge", "todo decision", actually are in no way related to the issue at hand.

Right now the GUI updater in Qubes doesn't work. We need to make it work. "Work" is defined as display updates when there are updates, and attempt to update those packages.

GUI updater works in Debian, Fedora, Ubuntu, Kubuntu, Linux Mint, etc. Is it an issue with how Qubes interacts with Apper that can be fixed? Then let's fix whatever isn't working. Is it an issue with Apper itself that cannot be fixed? Then it sounds like the program doesn't do its intended functionality and it shouldn't exist. So then we can switch to another program.

Yes there are problems with exit codes, automation etc. Those seem to be more general issues with updaters, not related/intrinsic to GUI updaters. That is a longer term project. This issue is / should be about making Apper work in Qubes (just as poorly as it does in other distros).

Your criticism and attempt to move this forward is well appreciated! Thanks for posting qubes issue Templates can't be updated via GUI Package Updater / Apper!

The worst case scenario is defined as noticing that updates are missing while thinking that updates have been applied cannot be fixed with a quick fix.

Right now the GUI updater in Qubes doesn't work. We need to make it work. "Work" is defined as display updates when there are updates, and attempt to update those packages.

GUI updater works in Debian, Fedora, Ubuntu, Kubuntu, Linux Mint, etc. Is it an issue with how Qubes interacts with Apper that can be fixed? Then let's fix whatever isn't working. Is it an issue with Apper itself that cannot be fixed?

It's both.

Then it sounds like the program doesn't do its intended functionality and it shouldn't exist. So then we can switch to another program.

I don't think working alternatives to Apper are available for installation. Definition would not be fulfilled for the display updates when there are updates part for a big percentage of users.

Yes there are problems with exit codes, automation etc. Those seem to be more general issues with updaters, not related/intrinsic to GUI updaters. That is a longer term project. This issue is / should be about making Apper work in Qubes (just as poorly as it does in other distros).

Let's use Templates can't be updated via GUI Package Updater / Apper for the short time fix. And this ticket for the long time fix.


Until there is a long term fix (working, secure, usable gui updater), it boils down to a question of security vs usability. I only see two options:

  • a) The secure option with bad usability is fall back to cli updating until the long term fix is in place.
  • b) The insecure option with almost as bad usability (common worst case scenario) is the short time fix + Apper.

Unless there is a dedicated developer working on the long term fix, I am very pessimistic. There probably won't be any remotely great solution to this issue.

mfc added a comment.Sep 21 2015, 4:17 PM

I think one workaround to bridge short-term & long-term may be to have the App Menu listing just be a shortcut to a terminal that runs apt-get update && apt-get upgrade (or yum update).

We can't expect the user to be able to type anything in the command line, but y/n for prompts I think is fine for a workaround.

A starting point to run a cli or GUI updater could be achieved with a minor modification in msgcollector.

Instead of displaying whonixcheck results systematically, parse the message from msgdispatcher.

  • result OK (all green) -> do nothing
  • warning or error -> display a tray icon with a menu offering to show the error only, or the whole message, and (if apt-get reports that packages can be updated) an option to run apt-get update && apt-get upgrade in a terminal, for the time being. The tray icon being closed after the user close the GUI or the terminal.

Even if an automatic upgrade in a terminal is not wanted, this approach would have the benefit of being less intrusive than the compulsory popup.

Patrick removed nrgaway as the assignee of this task.Nov 12 2018, 8:43 AM
Patrick closed this task as Resolved.Nov 12 2018, 8:47 AM
Patrick claimed this task.

Apper no longer installed by default.