This chapter describes the function and configuration of the
desktop server. The desktop server is responsible for storing
icon and application data for the entire HELIOS server,
and for managing file and directory IDs for network
volumes. The desktop server is usually invisible to users
and has no associated client application. The exception is
the Rebuild Desktop option in HELIOS Admin or the
“rebuild” tool (see 8.12 “rebuild”).
The following features have been added by the new SQL desktop database:
The new desktop database is compatible with all services/ tools. EtherShare, PCShare, WebShare and their connected clients will not see any differences using the new SQL desktop database.
Power failures or unexpected reboots do not require a desktop file rebuild during startup. All existing volumes are available within seconds.
The new desktop file is compatible between different host byte orders, e.g. between Solaris SPARC and Linux systems. Access from systems with different host byte orders is possible, without any need to rebuild the desktop file for each byte order. Hence, a Linux archive solution can mount an NFS volume from a Solaris SPARC main server, and use the HELIOS “dt” tools without any byte order issues.
The previous desktop database file was limited to a file size of 2 GB per volume which allowed about 25 million files per volume, depending on the file name length. Now there is no file size restriction anymore.
After 60 seconds of idle time (no file changes, only read-only access) the desktop database is flushed.
The live running desktop can be locked and copied into a backup directory (see 13.2.1 “Online backup copies of a live volume”).
The desktop database is flushed
via srvutil reconf desksrv.
Desktop flush and a snapshot file system copy are possible (depending on the operaring system and file system, respectively). This allows continuing with
the live volume read/write and using the snapshot data as a read-only volume.
The new SQL desktop format is default for all volumes with read/write access. Read-only file systems, e.g. CD-ROMs, with an old desktop format are detected and the old format will be used.
Unique file/folder IDs
Unique file and folder IDs are allocated when the file or folder is first created, and never change even if the file is renamed or moved to another location in the same volume
Renaming a file/folder will not destroy references (aliases, file IDs, OPI links, etc.) because identification is handled via the unique file/folder ID
Mac aliases can always be resolved to the linked file/folder
Mac and Windows compatibility
The desktop database is compatible with AFP/SMB protocols and HFS/NTFS file systems
Fast “find file” searching
A file or folder search takes only seconds vs. hours (no scanning of files/folders needed)
Future metadata support
The desktop server system consists of the program “desksrv”. It is created in the “HELIOSDIR/sbin” directory during the installation. HELIOS Base is configured to start “desksrv” automatically when the server is booted.
“desksrv” manages a database containing icon and application data for the entire HELIOS server, and file and directory ID information for network volumes. The database data is contained in the “.Desktop” file in the root directory of each HELIOS volume. The file name “.Desktop” is protected by all HELIOS servers, and is usually invisible to Mac, Windows or web users.
The application data records in each “.Desktop” file contain the information required to assign documents to the Mac applications that created them. The Mac Finder uses this information to start the corresponding application when the user double-clicks on a particular document. Each time a new application is copied into a HELIOS volume, “desksrv” creates a new entry in the corresponding desktop database file.
The icon data records in each “.Desktop” file contain icons from all the applications in the volume, sorted by type and creator codes.
When the Finder or an application needs to open a file, in addition to looking for the specified file name in a named directory and volume, a system option is available to open the file by a reference number instead. This reference number is called the “File ID”. The directory path can be specified by its “Folder ID”, too. By using file and folder IDs, files can still be found after a user has renamed or moved them. This provides users with great flexibility.
This principle is used in the Mac “Alias” facility.
Each “.Desktop” file contains file and folder IDs for all files and directories in the volume. The file ID is also stored in each file’s resource file; the folder ID is also stored in each directory’s resource file.
The file and folder IDs are allocated when the file or directory is first created, and never change even if the file is renamed or moved to another location in the same volume. The IDs are normally invisible to the user.
To rebuild the desktop of a HELIOS volume, you must
make sure that the volume is not mounted (e.g. with the
Active Users option described in 7.5.6 “Active users”
and then start HELIOS Admin on one of your
clients, select the volume in the volume list, and choose
Rebuild Desktop from the Volume
menu (see 7.6 “Volume menu”).
The rebuild process will always add files to the desktop database, to allow file name searches in the volume desktop database. This requires that a resource for every file/folder is available.
During a desktop rebuild, missing resource directories and resource files are automatically created for all directories in the HELIOS volume (including the volume root directory); existing directory IDs are verified and missing ones are created.
Also, missing file IDs are created and existing file IDs are verified. Missing resource files are created as well. The file type in that case is substituted by a dummy, which can be e.g. “UNIX/UNKN”. The generic UNIX icons and extension-mapped icons are created later by the File server, when the corresponding folder is first opened.
By default, the “rebuild” program uses two passes to update the volume desktop database. The two pass approach does have the benefit that on volumes where duplicate file/folder IDs exist (usually resulting from manipulations of files/ folders by UNIX commands), files/folders will get an adjusted ID only after the complete desktop is verified, and all other objects will retain their original ID. File and directory IDs are stored in both the “.Desktop” file and the file’s (or directory’s) resource file; if the verification detects differences, the IDs from the resource files are assumed to be the correct ones.
While a rebuild is running, it is possible to write to a HELIOS volume. As long as the desktop database is not yet complete, changes to files or folders may result in different file/folder IDs issued for objects not yet re-enumerated by “rebuild”.
To prevent inconsistencies between information stored in the desktop database of a HELIOS volume and the files/ folders in this volume, the rebuild process encountering a communication timeout will exit immediately. Desktop information for this volume will be incomplete until a rebuild finished successfully.
In the case of files created by UNIX applications, the resource directories are unnecessary – although they do not affect UNIX applications which access the original files, they take up a certain amount of disk space. Accordingly, it may be better to organize your directories and volumes with separate file systems for UNIX only and Mac, Windows or web files.
We strongly recommend to use the “rebuild” program
to put your desktop in order (see 8.12 “rebuild”) or to
use the Rebuild Desktop feature in HELIOS Admin.
Never delete the desktop database under UNIX!
Each time you open a HELIOS volume with the Connect to
Server dialog, the desktop server checks the logical
consistency of the desktop file and may trigger a desktop
rebuild if it finds invalid information. This process takes place
automatically and in the background. The same check is
done also for each volume – according to the setting of the
AFPPublish and SMBPublish volume flags in the
“Preferences” file – when HELIOS is first started.
In case “afpsrv” processes also encounter timeouts, the
volume will get secured against write access and will be
grayed out in the Connect to Server volume list. Opening
folders and reading files will still be possible, but creating/
renaming/moving files or folders will not be possible any
longer. Not even trashing files/folders will be allowed.
Timeouts are symptoms which are usually caused by high server or I/O load, slow I/O devices, network or communication problems. Assure that the cause of this problem is solved before restarting HELIOS services. Depending on the severity of the problem, it may be necessary to stop and restart HELIOS services on the server by using the scripts “HELIOSDIR/bin/stop-helios now” followed by “HELIOSDIR/bin/start-helios”.
In the case of CD-ROM volumes which have been mounted with HELIOS Admin as “read-only”, the desktop server first checks to see if a valid “.Desktop” file is available on the volume’s root. This can be the case, for example, for MO cartridges, which were previously mounted under EtherShare as “read-write”, allowing a valid desktop to be created.
If a valid “.Desktop” file cannot be found (this will almost always be the case for CD-ROMs), the desktop server creates a temporary desktop file in the host’s “/tmp” directory using a unique file name starting with “desksrv…”. Since the files on the CD-ROM almost certainly have no resource forks in this case, the Finder will use the information that are available in the extension mapping table or show one of the about 20 generic EtherShare icons.
Regularly backing up volumes that contain sensitive data is a must. In doing so, the desktop database is also saved. However problems may arise if the desktop database is being changed while it is saved, due to file changes by the HELIOS file servers. When restoring the backup to the volume, the database may be inconsistent, which will lead to failures or error messages. One workaround is to make the volume inactive during the time span of the backup.
A quite better method is to freeze the desktop for a while
so snapshots of the volume can be done. The desktop server supports flushing with a following 10 seconds freeze (default) or with a duration specified via the srvmsg -c -n desksrv freeze [duration] command. This simplifies doing snapshots. The new server preferences FreezeDefault and FreezeMax allow specifying the default freeze duration and the maximum freeze duration, respectively.
You can do an online “.Desktop” backup copy with the “backup-desktop.pl”
script (command line via perl backup-desktop.pl).
The script can be found on the HELIOS website:
www.helios.de/web/EN/support/desktop_backup.html
Do not set the value for FreezeMax unnecessarily high. Otherwise a client RPC timeout error may occur and the clients try to connect anew.
File changes in HELIOS volumes that were done without using EtherShare, PCShare, WebShare, or the HELIOS “dt” tools require a desktop rebuild in order to maintain a proper volume index. The workflow should be checked to eliminate any file operations that bypass the HELIOS processes. Problems to look for include host scripts not using the “dt” tools, FTP uploads, improper restoring of volume data from a system backup (see 8.11.3 “Using “dt” for backup/restore”), and other non-HELIOS compatible file changes. Note that an up-to-date desktop database is required for EtherShare and PCShare clients because they access files via unique IDs. Wrong or missing IDs will result in major problems.
If a UNIX error message like the following is logged:
missing DB shutdown:[/apple1/software] at Thu Nov 24 18:22:48 2009, recommend rebuild to re-create desktop file
a rebuild of this desktop database is recommended on the next possible occasion.
Note that the new transaction database does not need any rebuild to go online after an abort such as power failure.
Using non-HELIOS desktop compatible file services, e.g. Samba on a live HELIOS volume will cause problems such as jumping Finder listings, strange “File not found…” messages, etc.
Create an entirely new desktop:
# rebuild -f /volpath (as used in HELIOS Admin)
(No data loss because each file's ID is stored in its corresponding resource file, which will be indexed)
Re-index the desktop (missing/wrong file index will be repaired):
# rebuild /volpath
Scan the file system against the desktop database:
# rebuild -s /volpath
No output means that the index is up-to-date.
We strongly recommend that you check that sufficient free space is available on the disk before rebuilding the desktop (see 8.12 “rebuild”)!
By using the desktoplocation preference, a HELIOS volume desktop database can be relocated to a storage location outside a HELIOS volume.
This can have the following advantages:
Avoid problems when the volume runs out of free disk space
Performance enhancement using the desktop database on an idle disk
When this preference is set, volumes will not be automatically rebuilt. All volumes will keep their existing volume desktop databases, unless a rebuild is issued explicitly.
Only a rebuild in HELIOS Admin or using the rebuild -f command
will re-create the “.Desktop” file in its new location. The old
“.Desktop” file will be turned into a symlink to the new location.
The symlink can also be modified to point to a different custom location.
If a volume is used also remotely via NFS by the “dt” tools, make sure that both volume data and “.Desktop” file are accessible over NFS at the path specified. Otherwise you could end up with a separate volume desktop which is only current on the second system where “dt” is used to access volume data over NFS.
Preference example with relative path (the existing volumepath is /data/helios):
# prefvalue -k Programs/desksrv/desktoplocation -t str "var/desktops"
This will create the volume “.Desktop” file in “HELIOSDIR/var/desktops/data/helios/.Desktop”.
Preference example with absolute path (the existing volume path is “/data/helios”):
# prefvalue -k Programs/desksrv/desktoplocation -t str "/desktops"
This will create the volume “.Desktop” file in “/desktops/data/helios/.Desktop”.
After the preference is set, “desksrv” must be restarted.
The SQL desktop uses the underlying SQLite database file format. The command line tool “sqlite” (see 8.9 “sqlite”) can be used to administer the SQL desktop database.
The Mac OS X 10.4 or any other “SQLite” distribution is incompatible with the HELIOS SQL desktop version, therefore non-HELIOS delivered “sqlite” tools or libraries will not work or will result into locking problems, which will damage the database. It is safe to use the HELIOS included “sqlite” command.
The SQLite documentation can be found here:
www.sqlite.org/docs.html
Modifying the SQL desktop database should only be done by experienced developers who know about SQL and the file/directory ID requirements within the HELIOS products. Locking the database for a long period (longer than a few seconds) may result in network time-outs of the remotely connected Mac or Windows clients. If clients attempt to change volume information and the desktop database file of that volume is locked, you will see a warning message like: “Warning, desktop DB busy by another process”.
The HELIOS “desksrv” service will prepare changes into a
transaction and flushes this every 60 seconds (default
value; see SQLFlushDelay in 19.5 “Desktop server preference keys”),
to ensure that the SQL desktop file contains
the latest information. The srvutil reconf desksrv
command will issue a flush command for volumes.
The old desktop format is only supported to grant access to old read-only volume data. Therefore we recommend not to set or change this preference. The default – if not set – is the new SQL desktop format. For read-only file systems the desktop server will automatically detect and use the old format.
“afpsrv” or “pcshare” will try to connect to the desktop server “desksrv” to retrieve or store information from/to the desktop database of its volumes.
If the server does not get a response from “desksrv” in 60 seconds at longest, this indicates a severe problem with the underlying server operating system, UNIX file system or RAID/HSM drivers, or a hardware problem related to storage devices.
To prevent inconsistencies between information stored in the desktop database of a HELIOS volume and the files/folders in this volume, the “afpsrv” client process encountering this error will secure this volume (all mounted volumes) against write access. Opening folders and reading files will still be possible, but creating/renaming/moving files or folders will not be possible any longer. Not even trashing files/folders will be allowed.
Apart from the usual messages issued in the host system error log, the server will issue an error message on the client if the communication to “desksrv” fails, and inform users that the volume falls back to “read only” access. The message is:
Volname: localhost: RPC: Timed out; Only readonly access possible
Under certain conditions, an additional error message is issued and the volume is unmounted automatically by the Finder.
Though this message is issued by one “afpsrv” client process only, it is likely that all other “afpsrv” processes may encounter the same problem and therefore, the UNIX system administrator should be notified immediately.
Since applications usually work with temporary files/folders,
it may not be possible to save single files. If this problem
was only temporary, unmounting all HELIOS volumes
and remounting them will make these volumes writable
again. If this was not a temporary problem and is not
solved, then the Mac Connect to Server
dialog will show no volumes at all. Not even “root” will be able to
mount any of these volumes.
Depending on the severity of the problem, it may be necessary to stop and restart the HELIOS services on the server by using the scripts “bin/stop-helios now” followed by “bin/start-helios”.
Make sure that the cause of this problem is solved before restarting the HELIOS services.