Jump to page content

Packages

Caveats

Most if not all pages are just rough notes, and these pages as a whole are far from complete. More notes will be added in time, eventually, maybe.

If, from reading these notes, you conclude that I am off my rocker, you won’t be the first, and you may even be right. These pages may simply position me as an acolyte to the late Gene Ray.

No doubt there are a dozen and one reasons why none of this would ever work, but perhaps somewhere deep down there is a tiny fragment that could be used for something.

Contents

Overview

All software packages would be static and manifest-driven. A package is simply the directory inside an archive file, with a manifest file defining the package information. Installation of the package involves the package file being uncompressed to the Packages directory, after which the manifest is read and the relevant processing applied, e.g. registration of configuration slice definitions, registration of supported file types. The Multidimensional Filing System makes all the internal files available in their respective dimensions. Daemons are ratified automatically and made available for starting, and then started if needed, unless the manifest indicates that configuration must take place first in a supplied configuration utility included in the package.

Scriptable packages are too dangerous, as their activity is beyond the control of the operating system and thus beyond the control of the user. Package scripts are only reversible if the system goes to great lengths to record every action taken and provide rollback capability, and this is not practical.

File and protocol associations would be manifest driven as per the Macintosh. Layer Config slices would be observed directly. Any other system plugins such as for logging would be detected and obeyed as needed and as permitted.

Packages would allow nesting, e.g. the Foo Suite package could contain sub-packages Foo Writer and Foo Grid. This would allow the user to install and remove portions of the suite at the package manager level as well as to remove the entire suite collectively. Package nesting is completely distinct from package dependencies. Third-party depedencies could be nested if desired, or simply referenced to a central repository.

Installing a new version of a package should always leave at least one previous version of the package behind, at least for a while, so that the installation can be rolled back.

Management

Add/Remove Programs in Windows covers collectively application packages and dependency packages, but only very erratically. Installed PowerShell modules are not included, and there is no concept of suites: Microsoft Office appears as a single entry as the OS cannot manage sub-packages and depedencies at that level.

Linux users will be aware that Linux systems have vast numbers of installed packages that are far too numerous to cope with in a user-level Add/Remove Programs system. As such, there needs to be:

  1. Package suites that collect together all the related packages for a particular software system, and
  2. High-level and low-level views of installed packages, where the high-level view only shows applications

Suites

Packages can be collected together using suites, to make software management more straightforward. Suites are distinct from dependencies, as dependencies are often common and shared by multiple software suites. An example would be an office suite, containing a word processor, spreadsheet editor, desktop publishing application etc. Each would have its own package allowing independent installation and removal, but the entire suite could be collectively removed. Utility packages specific to this suite would sit inside the suite, while general dependencies would sit outside of it. The applications shipped with the OS itself (for example a calculator or a mail client) would share the OS’s built-in applications suite, not for the purposes of collective removal but rather to allow them to be collected together as a single unit when managing the software installled on a computer. Removal of individual programs from a suite would always be possible, but adding programs could only be done using the master package. Removal of hard dependencies from a suite would be prohibited; only soft dependencies (e.g. spelling checker, additional languages, example documents) could be removed.

Lots of products under Windows add multiple programs to Add/Remove Programs, and in each case, these should have been listed as a suite, except that Windows has never developed any such concept.

History

The original installation date of each package should be tracked, along with the date of the most recent update or upgrade and possibly even the dates of prior updates and upgrades. This allows the user (or administrator) to see not only when the package was updated (most recently or historically), but when the package was first installed, as these are useful diagnostics clues.

Within Windows at least, many applications are upgraded by way of first removing the old version and then installing the new version. This approach is not proscribed as it is not too dissimilar to the actual process, where the new version has its own directory and the OS rebuilds its caches of the package’s services. Regardless of the final package implementation, an update or “upgrade” (technically an upgrade means an improvement in product tier and is a term that is consistently abused) must not lose the original installation date or the history of that package’s updates.

Updates

The ability to keep multiple copies of a package simultaneously opens the possibility of allowing a program to be updated while it’s in use. Take the following example sequence:

  1. User A opens Foo Writer
  2. Foo Writer is updated, either automatically or as directed by a user or administrator; a new /Applications/Foo Writer is created alongside the existing one, with the new folder shadowing the old one, and user A continues to see the old package
  3. User B opens Foo Writer and sees the new version
  4. User A closes Foo Writer, releasing the package reference
  5. User A re-opens Foo Writer and now sees the new version

The implication here is that all the package containers (e.g. /Applications) are treated as dynamic maps: each time the user accesses a package, a dynamic map is created within their user session. When the package is deprecated and replaced, the user’s dynamic map continues to refer to the existing package directory. Once the user has closed all applications using that package, they will see the updates.

Library packages are versioned, and each version will be retained at least as long as any other installed package requires it. Launching a program that requires an older version of a library will automatically receive the older version according to its package manifest.

Having an application suddenly switch from an old version to a new version might be too alarming and confusing, especially for a piece of software that the user is opening and closing repeatedly during their work. One option would be to hold the new version until the user signs out. Another option would to be prompt the user whether or not to update immediately or after signing out; as interactive prompts are only appropriate in certain cases, where showing a prompt is impossible, the old version would by necessity be retained until sign-out.

Manifests

Each package’s manifest provides the package version along with all dependencies and the supported version of each. When starting an application, all library access requests will be automatically directed to the most recent accepted version of the library package. A library package cannot be removed while any active version of any application exists that depends on it. Inactive packages (those that are used solely by deprecated packages and that not part of any dynamic maps) can be removed, but all packages that depend on them will be removed in the process, which the user must agree to.

Conceivably the manifest could also specify the IPC command set version, such that if two programs communicate via the New Event Model, discrepancies can be identified by the operating system.

Application package manifests are covered in more detail on the applications page.

Repositories

See the repositories page.

Differential packages

Each package is a self-contained directory. Updating the software involves fetching a new package that is placed alongside the old one and supersedes it once nothing continues to reference the old package.

For large packages it would be ideal to support differential packages so that a product can be patched or updated without needing to replace a large package. Installation of a differential package would entail creation of a new package cloned from preceding version, plus any modified or new files in the differential package, minus any files marked in the differential package as obsolete. This is akin to copy-on-write in that the existing package is not touched: the differential process creates a new package as though the new package were retrieved in full and installed.

Differential packages are only valid against a specific subset of previous packages, and within a repository would likely only be maintained for the most recent version or versions.

Technically, the OS could even generate a reverse differential, i.e. it would move out of the existing package all files that are identical to the new version. The old package would become a differential downgrade package. Where the package size is large and the old version is not being used (no programs running that are using the package), this would offer a fallback option with much less disk space consumed. These reverse differentials could be used to allow package rollback in case of a software defect that renders the new version unusable.

Control

Part of the intention behind the Multidimensional Filing System dimensional projection is that packages will simply “exist”. A combination of the manifest plus the internal contents will drive all of what the package is created to achieve, with no programmatic code involved, allowing the OS total control and complete transparency and accountability. For example, daemons (services) will be identified directly by their existence and controlled by the package manifest and their own manifests (the latter will define their default behaviour, such as manual or automatic start).

This fulfils the robustness principle in that, should the system state become damaged, the package can be re-evaluated and returned to its default installation state, as all the data needed to restore operational state is contained inside the package and does not depend on any proprietary code to repair it.

As such, the only thing needed to uninstall a package is to simply delete it. Anything that the package was providing will simply be removed from the system (un-projected from MDFS and removed from all caches). Processes will be instructed to terminate and then killed.

In reality, there will be a lot of deeper questions to iron out to ensure that this works correctly. Most fundamentally, everything reasonable that installers are doing programmatically needs to be codified in the package manifest syntax to ensure that the design doesn’t prevent certain kinds of software being installed at all.

This system will, however, impose certain restrictions on what software vendors can do. It will be impossible to veto package removal: if the user or administrator wants the package gone, this will happen. The computer is not allowed to rule humans: humans rule the computer. With that said, removal of packages that serve as dependencies or as part of the system will be strongly resisted by the OS to avoid shooting oneself in the foot.

Identity

Packages require way to identify them and to reference other packages. Options include:

The identity of a package will be independent of its version. New versions will retain the same identity as older versions.

Where a package name needs to change, such as a rebranded product or a product acquired by another organisation, packages need to include aliases that allow references to older package identities to still resolve correctly. These are likely to be soft references only, as hard references (for hard dependencies) will likely change anyway as new versions of the dependency are released. Package aliases also allow existing configuration and file associations to transition across instances of packages being renamed. None of this will be required if the package identity is or contains a unique ID.

Packages will still require a human-readable name regardless, and this is how they will be presented. There is a risk of two packages having different internal identities but identical human-readable names, and thus a means is needed to differentiate them.