13 The desktop server

13.1 General remarks

This chapter describes the function and configuration of the desktop server. The desktop server is responsible for storing icon and application data for the entire HELIOS server, and for managing file and directory IDs for network volumes. The desktop server is usually invisible to users and has no associated client application. The exception is the Rebuild Desktop option in HELIOS Admin, and the “rebuild” tool respectively (see 8.11 “rebuild”).

13.1.1 The SQL desktop database

The SQL desktop database has the following features:

Transaction desktop file: Power failures or unexpected reboots do not require a desktop file rebuild during startup. All existing volumes are available within seconds.
Identical byte order: The desktop file is compatible with different host byte orders, e.g. with Solaris SPARC and Linux systems. Access from systems with different host byte orders is possible, without any need to rebuild the desktop file for each byte order. Hence, a Linux archive solution can mount an NFS volume from a Solaris SPARC main server, and use the HELIOS “dt” tools without any byte order issues.
Unlimited files per volume: The SQL desktop database has no file size restriction.
Auto flush/close: After 60 seconds of idle time (no file changes, only read-only access) the desktop database is flushed.
Online desktop backup copies/Snapshot desktop flush support: The live running desktop can be locked and copied into a backup directory (see 13.2.1 “Online backup copies of a live volume”). The desktop database is flushed via srvutil reconf desksrv. Desktop flush and a snapshot file system copy are possible (depending on the operating system and file system, respectively). This allows continuing with the live volume read/write and using the snapshot data as a read-only volume.
Backward compatibility: The SQL desktop format is default for all volumes with read/write access. Read-only file systems, e.g. CD-ROMs, with an old desktop format are detected and the old format will be used.

13.1.2 What is a desktop database good for?

Unique file/folder IDs
- Unique file and folder IDs are allocated when a file or folder is first created, and never change even if the file is renamed or moved to another location in the same volume
- Renaming a file/folder will not destroy references (aliases, file IDs, OPI links, etc.) because identification is handled via the unique file/folder ID
- Mac “Aliases” can always be resolved to the linked file/folder
Mac and Windows compatibility
- The desktop database is compatible with AFP/SMB protocols and HFS/NTFS file systems
Fast “find file” searching
- A file or folder search takes only seconds vs. hours (no scanning of files/folders needed)

13.2 The desktop server program

The desktop server system consists of the program “desksrv”. It is created in the “HELIOSDIR/sbin” directory during the installation. HELIOS Base is configured to start “desksrv” automatically when the server is booted.

desksrv

“desksrv” manages, on a per-volume basis, a database containing icon and application data for the entire HELIOS server, and file and directory ID information for network volumes. The database data is contained in the “.Desktop” file in the root directory of each HELIOS volume. The file name “.Desktop” is protected by all HELIOS servers, and is usually invisible to Mac, Windows or web users.

Mac OS 8/9 application and icon data

The application data records in each “.Desktop” file contain the information required to assign documents to the Mac applications that created them. The Mac Finder uses this information to start the corresponding application when the user double-clicks on a particular document. Each time a new application is copied into a HELIOS volume, “desksrv” creates a new entry in the corresponding desktop database file.

The icon data records in each “.Desktop” file contain icons from all the applications in the volume, sorted by type and creator codes.

File and folder IDs

When the Finder or an application needs to open a file, in addition to looking for the specified file name in a named directory and volume, a system option is available to open the file by a reference number instead. This reference number is called the “File ID”. The directory path can be specified by its “Folder ID”, too. By using file and folder IDs, files can still be found after a user has renamed or moved them. This provides users with great flexibility.

This principle is used in the Mac “Alias” facility.

Each “.Desktop” file contains file and folder IDs for all files and directories in the volume. The file ID is also stored in each file’s resource file; the folder ID is also stored in each directory’s resource file. The IDs are normally invisible to the user.

Desktop rebuild

To rebuild the desktop of a HELIOS volume, you must make sure that the volume is not mounted (e.g. with the Active Users option described in 7.5.10 “Online”).

: Start HELIOS Admin on one of your clients, select the volume in the volume list, and choose Rebuild Desktop from the Volume menu (see 7.6 “Volume menu”).

The rebuild process will always add files to the desktop database, to allow file name searches in the volume desktop database. This requires that a resource for every file/folder is available.

During a desktop rebuild, missing resource directories and resource files are automatically created for all directories in the HELIOS volume; existing directory IDs are verified and missing ones are created. Also, missing file IDs are created and existing file IDs are verified.

By default, the “rebuild” program uses two passes to update the volume desktop database. The two pass approach does have the benefit that on volumes where duplicate file/folder IDs exist (usually resulting from manipulations of files/ folders by UNIX commands), files/folders will get an adjusted ID only after the complete desktop is verified, and all other objects will retain their original ID. File and directory IDs are stored in both the “.Desktop” file and the file’s (or directory’s) resource file; if the verification detects differences, the IDs from the resource files are assumed to be the correct ones.

While a rebuild is running, it is possible to write to a HELIOS volume. As long as the desktop database is not yet complete, changes to files or folders may result in different file/folder IDs issued for objects not yet re-enumerated by “rebuild”.

To prevent inconsistencies between information stored in the desktop database of a HELIOS volume and the files/ folders in this volume, the rebuild process will exit immediately if it has encountered communication timeout. Desktop information for this volume will be incomplete until a rebuild finished successfully.

In the case of files created by UNIX applications, the resource directories are unnecessary – although they do not affect UNIX applications which access the original files, they take up a certain amount of disk space. Accordingly, it may be better to organize your directories and volumes with separate file systems for UNIX only and Mac, Windows or web files.

Important:: We strongly recommend to use the “rebuild” program to rebuild your desktop database (see 8.11 “rebuild”) or to use the Rebuild Desktop feature in HELIOS Admin. Never delete the desktop database under UNIX!

Desktop auto-rebuild

Each time you open a HELIOS volume with the Connect to Server dialog, the desktop server checks the logical consistency of the desktop file and may trigger a desktop rebuild if it finds invalid information. This process takes place automatically and in the background. The same check is done also for each volume – according to the setting of the AFPPublish and SMBPublish volume flags in the “Preferences” file – when HELIOS is first started.

Effects of timeouts

In case “afpsrv” and “pcshare” processes encounter timeouts with “desksrv”, the volume will get secured against write access and will be grayed out in the Connect to Server volume list. Opening folders and reading files will still be possible, but creating/renaming/moving files or folders will not be possible any longer. Not even trashing files/folders will be allowed.

Timeouts are symptoms which are usually caused by high server or I/O load, slow I/O devices, network or communication problems. The system log might help finding out what caused the timeout. Depending on the severity of the problem, it may be necessary to restart the HELIOS services on the server (“stop-helios now” then “start-helios”). However, assure that the cause of this problem is solved before restarting HELIOS services.

Read-only volumes

In the case of CD-ROM volumes which have been mounted with HELIOS Admin as “read-only”, the desktop server first checks to see if a valid “.Desktop” file is available on the volume’s root. This can be the case with e.g. MO cartridges that were previously mounted with EtherShare as “read-write”, allowing a valid desktop to be created.

If a valid “.Desktop” file cannot be found (this will almost always be the case for CD-ROMs), the desktop server creates a temporary desktop file in the host’s “/tmp” directory using a unique file name starting with “desksrv…”. Since the files on the CD-ROM almost certainly have no resource forks in this case, the Finder will use the information that are available in the extension mapping table or show one of the about 20 generic EtherShare icons.

13.2.1 Online backup copies of a live volume

Regularly backing up volumes that contain sensitive data is a must. In doing so, the desktop database is also saved. However problems may arise if the desktop database is being changed while it is saved, due to file changes by the HELIOS file servers. When restoring the backup to the volume, the database may be inconsistent, which will lead to failures or error messages. One workaround is to make the volume inactive during the time span of the backup.

A quite better method is to freeze the desktop for a while so snapshots of the volume can be done. The desktop server supports flushing with a following 10 seconds freeze (default) or with a duration that is specified via the command
srvmsg -c -n desksrv freeze [duration]. This simplifies doing snapshots. The server preferences FreezeDefault and FreezeMax allow specifying the default freeze duration and the maximum freeze duration, respectively.

You can do an online “.Desktop” backup copy with the “backup-desktop.pl” script (command line via perl backup-desktop.pl). The script can be found on the HELIOS website: www.helios.de/web/EN/support/desktop_backup.html

Note:: Do not set the value for FreezeMax unnecessarily high. Otherwise a client RPC timeout error may occur when the clients try to reconnect.

13.2.2 Rebuild the desktop database

Note:: Although it is not necessary to make a desktop rebuild after migration, it is advised to rebuild all desktops because we added a new database index which makes access faster.

File changes in HELIOS volumes that were done without using EtherShare, PCShare, WebShare, or the HELIOS “dt” tools require a desktop rebuild in order to maintain a proper volume index. The workflow should be checked to eliminate any file operations that bypass the HELIOS processes. Problems to look for include host scripts not using the “dt” tools, FTP uploads, improper restoring of volume data from a system backup (see 8.10.3 “Using “dt” for backup/restore”), and other non-HELIOS compatible file changes. Note that an up-to-date desktop database is required for EtherShare and PCShare clients because they access files via unique IDs. Wrong or missing IDs will result in major problems.

If a UNIX error message like the following is logged:

missing DB shutdown:[/apple1/software] at Thu Nov 24 18:22:48 2009,
recommend rebuild to re-create desktop file

a rebuild of this desktop database is recommended on the next possible occasion.

Note that the new transaction database does not need any rebuild to go online after an abort such as power failure.

Important:: Using non-HELIOS desktop compatible file services, e.g. Samba on a live HELIOS volume will cause problems such as jumping Finder listings, strange “File not found…” messages, etc.

a)

Create an entirely new desktop:

# rebuild -f /volpath (as used in HELIOS Admin)

(No data loss because each file's ID is stored in its corresponding resource file, which will be indexed)

b)

Re-index the desktop (missing/wrong file index will be repaired):

# rebuild /volpath

c)

Scan the file system against the desktop database:

# rebuild -s /volpath

No output means that the index is up-to-date.

Important:: We strongly recommend that you check that sufficient free space is available on the disk before rebuilding the desktop (see 8.11 “rebuild”)!

13.2.3 Relocate the desktop databases

By using the desktoplocation preference, a HELIOS volume desktop database can be relocated to a storage location outside a HELIOS volume.

This can have the following advantages:

Avoid problems when the volume runs out of free disk space
Performance enhancement using the desktop database on an idle disk

When this preference is set, volumes will not be automatically rebuilt. All volumes will keep their existing volume desktop databases, unless a rebuild is issued explicitly.

Only a rebuild in HELIOS Admin or using the rebuild -f command will re-create the “.Desktop” file in its new location. The old “.Desktop” file will be turned into a symlink to the new location. The symlink can also be modified to point to an existing different custom location.

Note:: If a volume is used also remotely via NFS by the “dt” tools, make sure that both volume data and “.Desktop” file are accessible over NFS at the path specified. Otherwise you could end up with a separate volume desktop which is only current on the second system where “dt” is used to access volume data over NFS.

Example with a relative path (the existing volume path is “/data/helios”):

# prefvalue -k Programs/desksrv/desktoplocation -t str "var/desktops"

On the next “rebuild -f” command the volume “.Desktop” file in “HELIOSDIR/var/desktops/data/helios/.Desktop” will be created.

Example with an absolute path (the existing volume path is “/data/helios”):

# prefvalue -k Programs/desksrv/desktoplocation -t str "/desktops"

On the next “rebuild -f” command the volume “.Desktop” file in “/desktops/data/helios/.Desktop” will be created.

After the preference is set, “desksrv” must be restarted (see 14.1 “srvutil”).

If the target location for the symlink is not available, the error message OpenDesk <volume path>: Miscellaneous error is issued.

13.2.4 SQL desktop database engine

The SQL desktop uses the underlying SQLite database file format. The command line tool “sqlite” (see 8.8 “sqlite”) can be used to administer the SQL desktop database.

Important:: The OS X or any other “SQLite” distribution is incompatible with the HELIOS SQL desktop version, therefore non-HELIOS delivered “sqlite” tools or libraries will not work properly or will result into locking problems, which will damage the database. It is safe to use the HELIOS included “sqlite” command.

The SQLite documentation can be found here:
www.sqlite.org/docs.html

Note:: Modifying the SQL desktop database should only be done by experienced developers who know about SQL and the file/directory ID requirements within the HELIOS products. Locking the database for a long period (longer than a few seconds) may result in network timeouts of the remotely connected Mac or Windows clients. If clients attempt to change volume information and the desktop database file of that volume is locked, you will see a warning message like: “Warning, desktop DB busy by another process”.

The HELIOS “desksrv” service will prepare changes into a transaction and flushes this every 60 seconds (default value; see SQLFlushDelay in 19.6 “Desktop server preference keys”), to ensure that the SQL desktop file contains the latest information. The srvutil reconf desksrv command will issue a flushing for all volumes.

13.2.5 Note on the “UseSQL” preference

The old desktop format is only supported to grant access to old read-only volume data. Therefore we recommend not to set or change this preference. The default – if not set – is the SQL desktop format. For read-only file systems the desktop server will automatically detect and use the old format.

13.3 “afpsrv/pcshare” and “desksrv” coordination

“afpsrv” or “pcshare” will try to connect to the desktop server “desksrv” to retrieve or store information from/to the desktop database of its volumes.

If the server does not get a response from “desksrv” in 60 seconds at longest, this indicates a severe problem with the underlying server operating system, UNIX file system or RAID/HSM drivers, or a hardware problem related to storage devices.

To prevent inconsistencies between information stored in the desktop database of a HELIOS volume and the files/folders in this volume, the “afpsrv” or “pcshare” client process encountering this error will secure this volume (all mounted volumes) against write access. Opening folders and reading files will still be possible, but creating/renaming/moving files or folders will not be possible any longer. Not even trashing files/folders will be allowed.

Apart from the usual messages issued in the host system error log, the server will issue an error message on the client if the communication to “desksrv” fails, and inform users that the volume falls back to “read only” access. The message is:

Volname: localhost: RPC: Timed out; Only readonly access possible

Under certain conditions, an additional error message is issued and the volume is unmounted automatically by the Finder.

Though this message is issued by one “afpsrv” or “pcshare” client process only, it is likely that all other “afpsrv” processes may encounter the same problem and therefore, the server administrator should be notified immediately.

Since applications usually work with temporary files/folders, it may not be possible to save single files. If this problem was only temporary, unmounting all HELIOS volumes and remounting them will make these volumes writable again. If this was not a temporary problem and is not solved, then the Mac Connect to Server dialog will show no volumes at all. Not even “root” will be able to mount any of these volumes.

Depending on the severity of the problem, it may be necessary to stop and restart the HELIOS services on the server by using the scripts “stop-helios now” followed by “start-helios”.

Make sure that the cause of this problem is solved before restarting the HELIOS services.

HELIOS Manuals May 27, 2021