This section describes various aspects of security within Condor.
On a Unix system, UIDs (User IDentification numbers) form part of an operating system's tools for maintaining access control. Each executing program has a UID, a unique identifier of a user executing the program. This is also called the real UID. A common situation has one user executing the program owned by another user. Many system commands work this way, with a user (corresponding to a person) executing a program belonging to (owned by) root. Since the program may require privileges that root has which the user does not have, a special bit in the program's protection specification (a setuid bit) allows the program to run with the UID of the program's owner, instead of the user that executes the program. This UID of the program's owner is called an effective UID.
Condor works most smoothly when its daemons run as root.
The daemons then have the ability to switch their
effective UIDs at will.
When the daemons run as root,
they normally leave their effective UID and GID (Group IDentification)
to be those of user and group condor.
This allows access to the log files without
changing the ownership of the log files.
It also allows access to these files when
the user condor's home directory resides on an NFS server.
root can not normally access NFS files.
If there is no condor user and group on the system, an
administrator can specify which UID and GID the Condor daemons should
use when they do not need root privileges in two ways,
either with the CONDOR_IDS environment variable or the
CONDOR_IDS configuration file setting.
In either case, the value should be the UID integer, followed by a
period, followed by the GID integer.
For example, if a Condor administrator does not want to create a
condor user, and instead wants their Condor daemons to run as
the daemon user (a common non-root user for system daemons to
execute as), the daemon user's UID was 2, and group
daemon had a GID of 2, the corresponding setting in the Condor
configuration file would be CONDOR_IDS = 2.2.
On a machine where a job is submitted, the condor_ schedd daemon changes its effective UID to root such that it has the capability to start up a condor_ shadow daemon for the job. Before a condor_ shadow daemon is created, the condor_ schedd daemon switches back to root, so that it can start up the condor_ shadow daemon with the (real) UID of the user who submitted the job. Since the condor_ shadow runs as the owner of the job, all remote system calls are performed under the owner's UID and GID. This ensures that as the job executes, it can access only files that its owner could access if the job were running locally, without Condor.
On the machine where the job executes, the job runs either as the submitting user or as user nobody, to help ensure that the job cannot access local resources or do harm. If the UID_DOMAIN matches, and the user exists as the same UID in password files on both the submitting machine and on the execute machine, the job will run as the submitting user. However, if the user does not exist in the execute machine's password file and SOFT_UID_DOMAIN is True, then Condor will choose the submitting user's UID on the execute machine. If SOFT_UID_DOMAIN is False, and UID_DOMAIN matches, and the user is not in the execute machine's password file, then the job will run as user nobody.
Condor can also function on all platforms by starting up as user condor. Since user condor does not have the ability to switch UID or GID, all daemons run with both the UID and GID belonging to user condor. The condor_ shadow daemon and the job's executable also run as user condor. This has the effect that the job can access only the files and directories that are accessible to the user condor on the machine where the job was submitted. Owners of jobs must make their input readable to the user condor. A job's output must be placed in a directory that is writable by the user condor as well. In practice, this means creating world-writable directories for output from Condor jobs. This creates a potential security risk, in that any user on the machine where the job is submitted can alter the data, remove it, or do other undesirable things. It is acceptable in an environment where users can trust other users.
On platforms where root access is not needed, Condor can even function without a UID or GID of the user condor. A directory to act as the condor home directory is still required, containing the configuration files, spool, execute and log directories. This home directory is not technically the home directory of any user. In this case, a user condor may or may not even exist, but the directory is still referred to as the condor home directory. If the user condor does not exist, use the CONDOR_CONFIG environment variable such that all Condor daemons and tools can find their configuration file (which in turn defines the locations of other needed files and directories), or place a configuration file in /etc/condor/condor_config. The Condor daemons can then be started up by whatever UID and GID has access to the local condor directory. Normally, users without root access who wish to use Condor on their machines create a condor home directory somewhere within their own accounts and start up the daemons (to run with the UID of the user). As in the case where the daemons run as user condor, there is no ability to switch UIDs or GIDs. The daemons run as the UID and GID of the user who started them. On a machine where jobs are submitted, the condor_ shadow daemons all run as this same user. However, if other users on the machine are using Condor in this environment, the condor_ shadow daemons for these other users' jobs execute with the UID of the user who started the daemons. This is a security risk, since the Condor job of the other user has access to all the files and directories of the user who started the daemons. Some installations have this level of trust, but others do not. Where this level of trust does not exist, it is best to set up a condor account and group, or to have each user start up their own Personal Condor submit installation.
When a machine is an execution site for a Condor job, the Condor job executes with the UID of the user who started the condor_ startd daemon. This is also potentially a security risk, which is why we do not recommend starting up the execution site daemons as a regular user. Use either root or a user (such as the user condor) that exists only to run Condor jobs.
Under Unix, Condor runs jobs either as the user that submitted the jobs, or as the user called nobody. Condor uses user nobody if the value of the UID_DOMAIN configuration variable of the submitting and executing machines are different.
When Condor cleans up after a executing a vanilla universe job, Condor does the best that it can by deleting all of the processes started by the job. Unfortunately, it is possible to fool Condor, and leave processes behind after Condor has cleaned up. If the job is running as user nobody, it is possible for it to leave a lurker process lying in wait for the next job run as nobody. The lurker process may prey maliciously on the next nobody user job, wreaking havoc.
Condor could prevent this problem by simply killing all processes run by the nobody user, but this would annoy many system administrators. The nobody user is often used for non-Condor system processes.
Condor provides a two-part solution to this difficulty. First, create user accounts specifically for Condor to use instead of user nobody. These can be low-privilege accounts, as the nobody user is. Create one of these accounts for each virtual machine per computer, so that distinct users can be used for concurrent processes. This prevents malicious behavior between processes running on distinct virtual machines. Section 3.10.6 details virtual machines. For a sample machine with two virtual machines, create two users that are intended only to be used by Condor. As an example, call them nobody1 and nobody2. Tell Condor about these users with the VMx_USER configuration variables, where x is replaced with the virtual machine number. In this example:
VM1_USER = nobody1 VM2_USER = nobody2
Reconfigure Condor, so that Condor will make use of these users instead of the nobody user. One more change is required to prevent lurker processes: tell Condor that these accounts are intended only to be used by Condor, so Condor can kill all the processes belonging to these users upon job completion. The configuration variable EXECUTE_LOGIN_IS_DEDICATED is introduced and set to True for this purpose.
EXECUTE_LOGIN_IS_DEDICATED = TRUE
Notes:
Any executing process has a notion of its current working directory (cwd), the directory that acts as the base for all file system access. There are two sides to any Condor job: the submit side and the execution side. This implies that there are two cwds. On the submit side, the owner's cwd sets a default cwd as a job is submitted. The cwd can be changed with a command in the submit description file. Since many jobs can be submitted at the same time, the commands are flexible in order to set the cwd individually for each job if desired. This submit side cwd remains for the entire life of a job. The submit side cwd is also used as the cwd of the condor_ shadow daemon. Since file system access for the job goes through the condor_ shadow daemon, all accesses behave as if they were executing without Condor.
There is also a cwd associated with the Condor job on the execution machine. It is set to the execute subdirectory of Condor's home directory. This directory is world-writable, since a Condor job usually runs as user nobody. Normally, the executable would never access this directory, since all I/O system calls are passed back to the condor_ shadow daemon on the submit machine. However, in the event that the job that creates a core dump, the cwd on the execute machine needs to be accessible by the job so that it can write the core file. The core file is moved back to the submit machine, and the condor_ shadow daemon is informed. The condor_ shadow daemon sends e-mail to the job owner announcing the crash and providing a pointer to the core file, then residing in the submit side cwd.
While we strongly recommend starting up the Condor daemons as root, we understand that it is not always possible to do so. The main problems appear if you have one Condor installation shared by many users on a single machine, or if you are setting up machines to only execute Condor jobs. If you are setting up a submit-only installation for a single user, then there is no need for (or benefit from) running as root.
What follows are the effects on the various parts of Condor of running both with and without root access.
In addition, some system information cannot be obtained without root access on some platforms (such as load average on IRIX). As a result, when running without root access, the condor_ startd has to call other programs (for example, ``uptime'') to get this information. This is much less efficient than getting the information directly from the kernel (which is what we do if we're running as root). On Linux and Solaris, we can get this information directly without root access, so this is not a concern on those platforms.
If you can't have all of Condor running as root, at least consider whether you can install the condor_ startd as setuid root. That would solve both of these problems. If you can't do that, you could also install it as a setgid sys or kmem program (depending on whatever group has read access to /dev/kmem on your system) and that would at least solve the system information problem.
You might consider installing condor_ submit as a setgid condor program so that at least the stdout, stderr and UserLog files get created with the right permissions. If condor_ submit is a setgid program, it will automatically set it's umask to 002, so that creates group-writable files. This way, the simple case of a job that just writes to stdout and stderr will work. If users have programs that open their own files, they'll have to know to set the right permissions on the directories they submit from.
Condor provides support for strong authentication, encryption, integrity assurance, as well as authorization. Most of these security features are not visible to the user (one who submits jobs). They are enabled by site administrators through the use of configuration macros. This section describes the authentication, encryption, integrity assurance, as well as authorization configuration macros provided by Condor.
Authentication provides an assurance of the identity of one of the communicating parties. Mutual authentication provides an assurance of the identities of both of the communicating parties. Encoding information such that its contents is not easily decipherable by outsiders is called encryption. The integrity of a message is assured when any form of tampering with the message can be detected. With integrity support, nothing in the message can be added, deleted, or modified without being detected.
When Condor is installed, default configuration settings use no authentication, encryption, or integrity checks, nor are authorization checks provided. This allows newer versions of Condor with security features to work or interact with previous versions without security support. An administrator must modify the configuration settings to enable the security features.
Inside Condor, daemons need to communicate with each other; furthermore, various tools provided by Condor may also require communication with Condor daemons. All these communications can be made more secure through the proper usage of authentication, encryption, and integrity checks. Authorization can be used to protect resources in a Condor pool.
When a daemon receives a request, it uses the client's security configuration information together with its own configuration settings to decide upon the security aspects of the communication. This can be considered a negotiation between the client and the daemon. The daemon replies to the client with a set of reconciled policies that controls the communication, including authentication, encryption, and integrity algorithms.
If the daemon determines that authentication is required, then the client must follow the chosen authentication protocol. After the required authentication, the client can send its request to the daemon. The daemon identifies the access level required for the specific request, and it checks the configuration settings to determine if the client has the required access level. If the client has the required access level, permission is granted, and the request is serviced.
The configuration macro names follow a pattern. Each of the names starts with the string SEC_. This string is followed by a string that describes an access level. The levels are
DEFAULT
READ
WRITE
ADMIN
IMMEDIATE_FAMILY
CONFIG
OWNER
NEGOTIATOR
CLIENT
Both DEFAULT and CLIENT
from this list are not access levels.
The DEFAULT is used to define all levels of access
for a specific configuration variable when individual levels
are not specified.
The CLIENT is used to define the client's requirements
and preferences in a secure communication.
Still within the name of a configuration macro, the access level is followed by another underscore character and then a string describing the communication type. The communication types are
AUTHENTICATION
ENCRYPTION
INTEGRITY
NEGOTIATION
Two examples of the complete macro names are
SEC_ADMIN_AUTHENTICATION
and
SEC_DEFAULT_INTEGRITY.
Each configuration variable would be defined with one of four predefined values. The values are
REQUIRED
PREFERRED
OPTIONAL
NEVER
For example, a line in a daemon's configuration file
to require all interactions to be encrypted is
SEC_DEFAULT_ENCRYPTION = REQUIRED
A second example from a configuration file specifies that all
requests (from a client) that would require a WRITE
access level be authenticated is
SEC_WRITE_AUTHENTICATION = REQUIRED
A daemon uses both the client's security configuration together with its own configuration to choose the communication setting for authentication, encryption, or integrity check. The following table defines whether or not (Yes or No) a communication setting will be used, or if the setting cannot work (Fail) due to a mismatch in the configuration settings.
client daemon Yes/No/Fail
REQUIRED REQUIRED Yes
REQUIRED PREFERRED Yes
REQUIRED OPTIONAL Yes
REQUIRED NEVER Fail
PREFERRED REQUIRED Yes
PREFERRED PREFERRED Yes
PREFERRED OPTIONAL Yes
PREFERRED NEVER No
OPTIONAL REQUIRED Yes
OPTIONAL PREFERRED Yes
OPTIONAL OPTIONAL No
OPTIONAL NEVER No
NEVER REQUIRED Fail
NEVER PREFERRED No
NEVER OPTIONAL No
NEVER NEVER No
The client uses one of two macros to configure authentication:
SEC_DEFAULT_AUTHENTICATION
SEC_CLIENT_AUTHENTICATION
For the daemon, there are eight macros to configure authentication:
SEC_DEFAULT_AUTHENTICATION
SEC_READ_AUTHENTICATION
SEC_WRITE_AUTHENTICATION
SEC_ADMIN_AUTHENTICATION
SEC_IMMEDIATE_FAMILY_AUTHENTICATION
SEC_CONFIG_AUTHENTICATION
SEC_OWNER_AUTHENTICATION
SEC_NEGOTIATOR_AUTHENTICATION
As an example, the macro defined in the configuration file for a daemon as
SEC_WRITE_AUTHENTICATION = REQUIREDsignifies that the daemon must authenticate the client for any communication that requires the WRITE access level. If the daemon's configuration contains
SEC_DEFAULT_AUTHENTICATION = REQUIREDand does not contain any other security configuration for
AUTHENTICATION, then this default defines the daemon's needs
for authentication over all access levels.
Where a specific macro is present, its value takes
precedence over any default given.
If authentication is to be done, then the communicating parties must negotiate a mutually acceptable method of authentication to be used. A list of acceptable methods may be provided by the client, using the macros
SEC_DEFAULT_AUTHENTICATION_METHODS
SEC_CLIENT_AUTHENTICATION_METHODS
A list of acceptable methods may be provided by the daemon, using the
macros
SEC_DEFAULT_AUTHENTICATION_METHODS
SEC_READ_AUTHENTICATION_METHODS
SEC_WRITE_AUTHENTICATION_METHODS
SEC_ADMIN_AUTHENTICATION_METHODS
SEC_IMMEDIATE_FAMILY_AUTHENTICATION_METHODS
SEC_CONFIG_AUTHENTICATION_METHODS
SEC_OWNER_AUTHENTICATION_METHODS
SEC_NEGOTIATOR_AUTHENTICATION_METHODS
The methods are
given as a comma-separated list of acceptable values.
These variables list the authentication methods that are available
to be used.
The ordering of the list gives preference;
the first item in the list indicates the highest preference.
The values will be
KERBEROS
FS
CLAIMTOBE
ANONYMOUS
NTSSPI
GSI
As an example, the macro
SEC_DEFAULT_AUTHENTICATION_METHODS = KERBEROS, NTSSPIindicates that either Kerberos or Windows authentication may be used, but Kerberos is preferred over Windows.
The client uses one of two macros to enable or disable encryption:
SEC_DEFAULT_ENCRYPTION
SEC_CLIENT_ENCRYPTION
For the daemon, there are eight macros to enable or disable encryption:
SEC_DEFAULT_ENCRYPTION
SEC_READ_ENCRYPTION
SEC_WRITE_ENCRYPTION
SEC_ADMIN_ENCRYPTION
SEC_IMMEDIATE_FAMILY_ENCRYPTION
SEC_CONFIG_ENCRYPTION
SEC_OWNER_ENCRYPTION
SEC_NEGOTIATOR_ENCRYPTION
As an example, the macro defined in the configuration file for a daemon as
SEC_IMMEDIATE_FAMILY_ENCRYPTION = REQUIREDsignifies that any daemon to daemon communication must be encrypted. If a daemon's configuration contains
SEC_DEFAULT_ENCRYPTION = REQUIREDand does not contain any other security configuration for ENCRYPTION, then this default defines the daemon's needs for encryption over all access levels. Where a specific macro is present, its value takes precedence over any default given.
If encryption is to be done, then the communicating parties must find (negotiate) a mutually acceptable method of encryption to be used. A list of acceptable methods may be provided by the client, using the macros
SEC_DEFAULT_CRYPTO_METHODS
SEC_CLIENT_CRYPTO_METHODS
A list of acceptable methods may be provided by the daemon, using the
macros
SEC_DEFAULT_CRYPTO_METHODS
SEC_READ_CRYPTO_METHODS
SEC_WRITE_CRYPTO_METHODS
SEC_ADMIN_CRYPTO_METHODS
SEC_IMMEDIATE_FAMILY_CRYPTO_METHODS
SEC_CONFIG_CRYPTO_METHODS
SEC_OWNER_CRYPTO_METHODS
SEC_NEGOTIATOR_CRYPTO_METHODS
The methods are given as a comma-separated list of acceptable values. These variables list the encryption methods that are available to be used. The ordering of the list gives preference; the first item in the list indicates the highest preference. Possible values are
3DES
BLOWFISH
An integrity check assures that the messages between communicating parties have not been tampered with. Any change, such as addition, modification, or deletion can be detected. Through configuration macros, both the client and the daemon can specify whether an integrity check is required of further communication.
The client uses one of two macros to enable or disable an integrity check:
SEC_DEFAULT_INTEGRITY
SEC_CLIENT_INTEGRITY
For the daemon, there are eight macros to enable or disable an integrity check:
SEC_DEFAULT_INTEGRITY
SEC_READ_INTEGRITY
SEC_WRITE_INTEGRITY
SEC_ADMIN_INTEGRITY
SEC_IMMEDIATE_FAMILY_INTEGRITY
SEC_CONFIG_INTEGRITY
SEC_OWNER_INTEGRITY
SEC_NEGOTIATOR_INTEGRITY
As an example, the macro defined in the configuration file for a daemon as
SEC_IMMEDIATE_FAMILY_INTEGRITY = REQUIREDsignifies that any daemon to daemon communication must have its integrity assured. If a daemon's configuration contains
SEC_DEFAULT_INTEGRITY = REQUIREDand does not contain any other security configuration for
INTEGRITY, then this default defines the daemon's needs
for integrity checks over all access levels.
Where a specific macro is present, its value takes
precedence over any default given.
There is currently only one method used for integrity checking: a signed MD5 checksum. Its use is implied whenever integrity checks occur. If more methods are implemented, then there will be further macros to allow both the client and the daemon to specify which methods are acceptable.
A configuration file is provided when Condor is installed. No security features are enabled within the configuration as distributed. Included as comments within the configuration file is an example suggesting settings that enable security features. Here is that example of the daemon-side portion.
SEC_DEFAULT_AUTHENTICATION=REQUIRED SEC_DEFAULT_ENCRYPTION=REQUIRED SEC_DEFAULT_INTEGRITY=REQUIRED SEC_DEFAULT_AUTHENTICATION_METHODS = FS, KERBEROS SEC_DEFAULT_CRYPTO_METHODS = 3DES, BLOWFISH
This set of configuration macros forces security features to be used at all times. All communication is authenticated (using Kerberos), and all communication is both encrypted (using triple DES) and has its integrity checked to make sure that messages are not modified or corrupted. Negotiation configuration variables are not included in this suggested configuration. The default (when undefined) will be to set
SEC_DEFAULT_NEGOTIATION=OPTIONAL
Note that this example configuration requires that all Condor daemons be version 6.3.3 or later, since previous versions will not have the ability to do secure communication.
Where no authentication methods are specified in the configuration, the default is for Condor to use file system authentication, Kerberos authentication, or GSI authentication.
Several steps are required to enable this authentication.
PKI-based authentication between Condor daemons requires a valid X.509 certificate from a trusted CA (Certification Authority). The certificate may be a single host certificate, and all Condor daemons on the same machine may share the same certificate. The directory where the certificate resides, along with other related files such as one containing the private key, are utilized by Condor when doing authentication.
Condor locates the daemon's certificate through the use of a configuration macro. Note that the certificate can be shared by all Condor daemons running on a machine. The certificate can also be copied to other machines, where local copies are necessary. The certificates must be protected by access rights to files, since the password file is not encrypted.
Condor users (those who submit jobs) can also use the X.509-based authentication. These users must also have valid X.509 certificates signed by a trusted CA. For use when submitting jobs, the certificate is used to generate a proxy: a combination of certificate and password that is valid only for a specific (and usually short) time period. A proxy is used so that the user does not need to be present (to give a password) when a job is executed.
To use X.509 certificates for authentication, the Condor configuration file must have the following configuration macros set. This setting is an example.
SEC_DEFAULT_AUTHENTICATION = REQUIRED SEC_DEFAULT_AUTHENTICATION_METHODS = GSS_AUTHENTICATION X509_DIRECTORY = /path/to/daemon/certificatedirectory CONDOR_GATEKEEPER = /C=US/O=Condor/O=University of Wisconsin /OU=Computer Sciences Department/CN=condor@cs.wisc.edu
The SEC_DEFAULT_AUTHENTICATION macro specifies that authentication is required for all communications. This single macro covers all communications, but could be replaced with a set of macros that require authentication for only specific communications. See section 3.7.3 for details on the more specific configuration variables used for authentication.
The GSS_AUTHENTICATION method specifies the use of the X.509 protocol. If this is the only method listed, then X.509 will be the only protocol used. If other methods are acceptable, then placing this method first within the list will cause Condor to give preference to this method over others.
The macro X509_DIRECTORY must be specified, so that Condor knows where to look for the daemon's certificate. This path may be a directory or a shared file system such as AFS. Alternatively, this path name can point to local copies of the certificate stored in a local file system.
The macro CONDOR_GATEKEEPER configuration macro provides daemons with a distinguished name to use for X.509 authentication. This name is specified with the following format
CONDOR_GATEKEEPER = /C=?/O=?/O=?/OU=?/CN=<daemon_name@domain>A complete example that has the question marks filled in and the daemon's user name filled in is given in the example above.
Condor will also need a way to map an X.509 distinguished
name to a Condor user id.
This is done in an administrator-maintained file called an X.509 map file,
mapping from X509 Distinguished Name (DN) to Condor user id.
It is similar to a Globus Gridmap file.
Entries (lines) in the file each contain two items.
The first item in an entry is the
X.509 certificate subject, and it is enclosed in quotes
(using the character ").
The second item is the Condor user id, given as a
fully qualified name.
The two items in an entry are separated by tab or space character(s).
Here is an example of an entry in an X.509 map file.
Entries must be on a single line; this example is broken
onto two lines for formatting reasons.
"/C=US/O=Globus/O=University of Wisconsin/OU=Computer Sciences Department/CN=Alice Smith" asmith
Condor expects the map file to be in the location given by
$(X509_DIRECTORY)/certdir/index.txtIf the map file is not in this location, a symbolic link may be used to point to the actual map file.
The Condor user (one who submits jobs) must create a proxy prior to authentication. The user creates a proxy using the program grid-proxy-init. This program needs to know the location of the user's certificate directory. This is done by setting an environment variable, called X509_CERT_DIR. For example, if the X509 directory is /test/myX509, then X509_CERT_DIR should be set to /test/myX509/certdir.
After running grid-proxy-init, an environment variable called X509_USER_PROXY may be set to give the location where the proxy is stored. By default, grid-proxy-init will place the proxy in the /tmp directory with the file name being determined by the format:
/tmp/x509_uXXXXThe specific file name is given by substituting the
XXXX
characters with the UID of the user.
The
X509_USER_PROXY environment variable is optional,
since Condor looks in the default location if the environment
variable is not set.
KERBEROS_MAP_FILE = /path/to/etc/condor.kmap
Lines within the map file have the syntax
KERB.REALM = UID.domain.name
Here are two lines from a map file to use as an example:
CS.WISC.EDU = cs.wisc.edu ENGR.WISC.EDU = ee.wisc.edu
If a KERBEROS_MAP_FILE configuration variable is defined and set, then all permitted realms must be explicitly mapped. If no map file is specified, then Condor assumes that the Kerberos realm is the same as the Condor UID domain.
The configuration variable CONDOR_SERVER_PRINCIPAL defines the name of a Kerberos principal. If CONDOR_SERVER_PRINCIPAL is not defined, then the default value used is "host". A principal specifies a unique name to which a set of credentials may be assigned.
Condor takes the specified (or default) principal and appends a slash character, the host name, an '@' (at sign character), and the Kerberos realm. As an example, the configuration
CONDOR_SERVER_PRINCIPAL=condor-daemonresults in Condor's use of
condor-daemon/the.host.name@YOUR.KERB.REALMas the server principal.
Here is an example of configuration settings that use Kerberos for authentication and require authentication of all communications of the write or administrator access level.
SEC_WRITE_AUTHENTICATION = REQUIRED SEC_WRITE_AUTHENTICATION_METHODS = KERBEROS SEC_ADMINISTRATOR_AUTHENTICATION = REQUIRED SEC_ADMINISTRATOR_AUTHENTICATION_METHODS = KERBEROS
This form of authentication utilizes the ownership of a file in the identity verification of a client. A daemon authenticating a client requires the client to write a file in a specific location. The daemon then checks the ownership of the file. The file's ownership verifies the identity of the client. In this way, the file system becomes the trusted authority.
Authorization protects resource usage by granting or denying access requests made to the resources. It defines who is allowed to do what.
Authorization is defined in terms of users. An initial implementation provided authorization based on hosts (machines), while the current implementation relies on user-based authorization. Section 3.7.5 on Setting Up IP/Host-Based Security in Condor describes the previous implementation. This IP/Host-Based security still exists, and it can be used, but significantly stronger and more flexible security can be achieved with the newer authorization based on fully qualified user names.
Unlike authentication, encryption, and integrity checks, which can be configured by both client and server, authorization is used only by a server. The authorization portion of the security of a Condor pool is based on a set of configuration macros. The macros list which user/daemon will be authorized to issue what request given a specific access level.
These configuration macros define a set of users that will be allowed to (or denied from) carrying out various Condor commands. Each access level may have its own list of authorized users. A complete list of the authorization macros:
ALLOW_READ
ALLOW_WRITE
ALLOW_ADMINISTRATOR
ALLOW_CONFIG
ALLOW_IMMEDIATE_FAMILY
ALLOW_OWNER
ALLOW_NEGOTIATOR
DENY_READ
DENY_WRITE
DENY_ADMINISTRATOR
DENY_CONFIG
DENY_IMMEDIATE_FAMILY
DENY_OWNER
DENY_NEGOTIATOR
Each macro is defined by a comma-separated list of fully qualified users. Each fully qualified user is described using the following format:
username@domain/hostname
The information to the left of the slash character describes
a user within a domain.
The information to the right of the slash character describes
one or more machines from which the user would be issuing a command.
This host name may take the form of either a fully qualified host name
of the form
bird.cs.wisc.eduor an IP address of the form
128.105.128.0
An example is
zmiller@cs.wisc.edu/bird.cs.wisc.edu
Within the format, wildcard characters (the asterisk, *) are allowed. The use of wildcards is limited to one wildcard on either side of the slash character. A wildcard character used in the host name is further limited to come at the beginning of a fully qualified host name or at the end of an IP address. For example,
*@cs.wisc.edu/bird.cs.wisc.edurefers to any user that comes from
cs.wisc.edu,
where the command is originating from the machine
bird.cs.wisc.edu.
Another valid example,
zmiller@cs.wisc.edu/*.cs.wisc.edurefers to commands coming from any machine within the
cs.wisc.edu domain, and issued by zmiller.
A third valid example,
*@cs.wisc.edu/*refers to commands coming from any user within the
cs.wisc.edu domain
where the command is issued from any machine.
A fourth valid example,
*@cs.wisc.edu/128.105.*refers to commands coming from any user within the
cs.wisc.edu domain
where the command is issued from machines within the network that match
the first two octets of the IP address.
If the set of machines is specified by an IP address, then further specification using a net mask identifies a physical set (subnet) of machines. This physical set of machines is specified using the form
network/netmaskThe
network is an IP address.
The net mask takes one of two forms.
It may be a decimal number which refers to the number of leading
bits of the IP address that are used in describing a subnet.
Or, the net mask may take the form of
a.b.c.dwhere
a,
b,
c, and
d
are decimal numbers that each specify an 8-bit mask.
An example net mask is
128.255.0.4which specifies the bit mask
10000000.11111111.00000000.00000100
A single complete example of a configuration variable that uses a net mask is
ALLOW_WRITE = joesmith@cs.wisc.edu/128.105.128.0/17User
joesmith within the
cs.wisc.edu domain is given write authorization
when originating from machines that match their leftmost
17 bits of the IP address.
This flexible set of configuration macros could used to define conflicting authorization. Therefore, the following protocol defines the precedence of the configuration macros.
An example of the configuration variables for the user-side authorization is derived from the necessary access levels as described in Section 3.7.3.
ALLOW_READ = *@cs.wisc.edu/* ALLOW_WRITE = *@cs.wisc.edu/*.cs.wisc.edu ALLOW_ADMINISTRATOR = condor-admin@cs.wisc.edu/*.cs.wisc.edu ALLOW_NEGOTIATOR = condor@cs.wisc.edu/$(NEGOTIATOR_HOST) ALLOW_CONFIG = condor-admin@cs.wisc.edu/*.cs.wisc.edu ALLOW_IMMEDIATE_FAMILY= condor@cs.wisc.edu/*.cs.wisc.edu
This example configuration authorizes
any user in the
cs.wisc.edu domain to
carry out a request that requires the
READ access level
from any machine.
Any user in the
cs.wisc.edu domain may
carry out a request that requires the
WRITE access level
from any machine in the
cs.wisc.edu domain.
Only the user called condor-admin may
carry out a request that requires the
ADMINISTRATOR access level
from any machine in the
cs.wisc.edu domain.
Only the negotiator daemon, running as
condor on the machine defined by the
NEGOTIATOR_HOST macro is authorized
with the
NEGOTIATOR access level.
The administrator, logged into any machine within
the cs.wisc.edu domain is authorized at the
CONFIG access level.
And, the last line of the example presumes that there is a
user called condor, and that the daemons have all been started
up as this user.
It authorizes only programs (which will be the daemons)
running as
condor to
carry out requests that require the
IMMEDIATE_FAMILY access level,
where the commands originate from
any machine in the
cs.wisc.edu domain.
In the local configuration file for each host, the host's owner should be authorized as the owner of the machine. An example of the entry in the local configuration file:
ALLOW_OWNER = username@cs.wisc.edu/hostname.cs.wisc.eduIn this example the owner has a login of
username, and the machine's name is represented by
hostname.
This section describes the mechanisms for setting up Condor's host-based security. This is now an outdated form of implementing security at the level of machine access. It remains available and documented for purposes of backward compatibility. If used at the same time as the user-based authorization, the two specifications are merged together.
The host-based security allows control over what machines can join a Condor pool, what machines can find out information about your pool, and what machines within your pool can perform administrative commands. By default, Condor is configured to allow anyone to view or join your pool. You probably want to change that.
This section discusses how the host-based security works inside Condor. It lists the different levels of access and what parts of Condor use which levels. There is a description of how to configure your pool to grant (or deny) certain levels of access to various machines. Configuration examples and the settings of configuration variables using the condor_ config_val command complete this section.
Inside the Condor daemons or tools that use DaemonCore (see section 3.8 for details), most things are accomplished by sending commands to another Condor daemon. These commands are formed from an integer to specify which command, followed by any optional information that the protocol requires at that point (such as a ClassAd, capability string, etc). When the daemons start up, they register which commands they are willing to accept, what to do with arriving commands, and the access level required for that command. When a command arrives, Condor identifies the access level required, and checks the IP address of the sender to be sure it passes the various allow/deny settings in the configuration file for the given access level. If permission is granted, the command continues. If not, the command is aborted.
As expected, settings for the access levels in the global configuration file affect all the machines in the pool. Settings in a local configuration file only affect the specific machine. The settings for a given machine determine what other hosts can send commands to that machine. So, if machine foo is to be given administrator access on machine bar, place foo in bar's configuration file access list (not the other way around).
The following are the various access levels that commands within Condor can be registered with:
IMPORTANT: For a machine to join a Condor pool, the machine must have both WRITE permission AND READ permission. WRITE permission is not enough.
IMPORTANT: This access is given to a machine, and it applies to an entire pool. So, ADMINISTRATOR access for a given machine provides ANY USER on that machine ADMINISTRATOR rights (including users who can run Condor jobs on that machine). Therefore, grant ADMINISTRATOR access carefully.
Starting with version 6.3.2, Condor provides a mechanism for more fine-grained control over the configuration settings that can be modified remotely with condor_ config_val.
Host-based security access permissions are specified in configuration files.
ADMINISTRATOR and NEGOTIATOR access default to the central manager machine. OWNER access defaults to the local machine, as well as any machines given with ADMINISTRATOR access. CONFIG access is not granted to any machine as its default. These defaults work well, and should not be changed without a compelling reason. If machines other than the default are to have to have OWNER access, they probably should also have ADMINISTRATOR access. By granting machines ADMINISTRATOR access, they will automatically have OWNER access, given how OWNER access is set within the configuration.
The default access configuration is
HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST) HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR) HOSTALLOW_READ = * HOSTALLOW_WRITE = * HOSTALLOW_NEGOTIATOR = $(NEGOTIATOR_HOST) HOSTALLOW_NEGOTIATOR_SCHEDD = $(NEGOTIATOR_HOST), $(FLOCK_NEGOTIATOR_HOSTS) HOSTALLOW_WRITE_COLLECTOR = $(HOSTALLOW_WRITE), $(FLOCK_FROM) HOSTALLOW_WRITE_STARTD = $(HOSTALLOW_WRITE), $(FLOCK_FROM) HOSTALLOW_READ_COLLECTOR = $(HOSTALLOW_READ), $(FLOCK_FROM) HOSTALLOW_READ_STARTD = $(HOSTALLOW_READ), $(FLOCK_FROM)
For each access level, an ALLOW or a DENY may be added.
Multiple machine entries in the configuration files may be separated by either a space or a comma. The machines may be listed by
To resolve an entry that falls into both allow and deny: individual machines have a higher order of precedence than wildcard entries, and host names with a wildcard have a higher order of precedence than IP subnets. Otherwise, DENY has a higher order of precedence than ALLOW. (this is how most people would intuitively expect it to work).
In addition, the above access levels may be specified on a per-daemon basis, instead of machine-wide for all daemons. Do this with the subsystem string (described in section 3.3.1 on Subsystem Names), which is one of: STARTD, SCHEDD, MASTER, NEGOTIATOR, or COLLECTOR. For example, to grant different read access for the condor_ schedd:
HOSTALLOW_READ_SCHEDD = <list of machines>
The following is a list of registered commands that daemons will accept. The list is ordered by daemon. For each daemon, the commands are grouped by the access level required for a daemon to accept the command from a given machine.
ALL DAEMONS:
The command sent as a result of condor_ reconfig to reconfigure a daemon.
The command sent as a result of reconfig -full to perform a full reconfiguration on a daemon.
STARTD:
All commands that relate to a condor_ schedd daemon claiming a machine, starting jobs there, or stopping those jobs.
The command that condor_ checkpoint sends to periodically checkpoint all running jobs.
The command that condor_ preen sends to request the current state of the condor_ startd daemon.
NEGOTIATOR:
COLLECTOR:
SCHEDD:
The commands that a condor_ startd sends to the condor_ schedd when it must vacate its jobs and release the condor_ schedd's claim.
The commands which write information into the job queue (such as condor_ submit and condor_ hold). Note that for most commands which attempt to write to the job queue, Condor will perform an additional user-level authentication step. This additional user-level authentication prevents, for example, an ordinary user from removing a different user's jobs.
MASTER: All commands are registered with ADMINISTRATOR access:
This section provides examples of configuration settings. Notice that ADMINISTRATOR access is only granted through a HOSTALLOW setting to explicitly grant access to a small number of machines. We recommend this.
HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST) HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
HOSTALLOW_READ = *.ncsa.uiuc.edu HOSTALLOW_WRITE = *.ncsa.uiuc.edu HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST) HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
HOSTALLOW_WRITE = *.ncsa.uiuc.edu, *.math.uiuc.edu HOSTDENY_WRITE = lab-*.edu, *.lab.uiuc.edu, 177.55.* HOSTALLOW_ADMINISTRATOR = bigcheese.ncsa.uiuc.edu HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
\'' to continue a long list of machines
onto multiple lines, making it more readable (this works for all
configuration file entries, not just host access entries)
HOSTALLOW_READ = *.ncsa.uiuc.edu, *.cs.wisc.edu
HOSTALLOW_WRITE = *.ncsa.uiuc.edu, raven.cs.wisc.edu
HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST), bigcheese.ncsa.uiuc.edu, \
biggercheese.uiuc.edu
HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
HOSTDENY_READ = *.mil
HOSTALLOW_READ_SCHEDD = *.ncsa.uiuc.edu
HOSTALLOW_WRITE = *.ncsa.uiuc.edu
HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST), bigcheese.ncsa.uiuc.edu, \
biggercheese.uiuc.edu
HOSTALLOW_ADMINISTRATOR_NEGOTIATOR = biggercheese.uiuc.edu
HOSTALLOW_OWNER = $(FULL_HOSTNAME), $(HOSTALLOW_ADMINISTRATOR)
A new security feature introduced in
Condor version 6.3.2 enables more fine-grained control over the
configuration settings that can be modified remotely with the
condor_ config_val command.
The manual page for condor_ config_val on
page ![[*]](crossref.png)
details how to use
condor_ config_val to modify configuration settings remotely.
Since certain configuration attributes can have a large impact on the
functioning of the Condor system and the security of the machines in a
Condor pool, it is important to restrict the ability to change
attributes remotely.
For each security access level described, the Condor administrator can define which configuration settings a host at that access level is allowed to change. Optionally, the administrator can define separate lists of settable attributes for each Condor daemon, or the administrator can define one list that is used by all daemons.
For each command that requests a change in configuration setting, Condor searches all the different possible security access levels to see which, if any, the request satisfies. (Some hosts can qualify for multiple access levels. For example, any host with ADMINISTRATOR permission probably has WRITE permission also). Within the qualified access level, Condor searches for the list of attributes that may be modified. If the request is covered by the list, the request will be granted. If not covered, the request will be refused.
The default configuration shipped with Condor is exceedingly restrictive. Condor users or administrators cannot set configuration values from remote hosts with condor_ config_val. Enabling this feature requires a change to the settings in the configuration file. Use this security feature carefully. Grant access only for attributes which you need to be able to modify in this manner, and grant access only at the most restrictive security level possible.
The most secure use of this feature allows Condor users to set
attributes in the configuration file which are not used by Condor
directly.
These are custom attributes published by various Condor
daemons with the SUBSYS_EXPRS setting described in
section 3.3.4 on page .
It is secure to grant access only to modify attributes that are used by Condor
to publish information.
Granting access to modify
settings used to control the behavior of Condor is
not secure.
The goal is to
ensure no
one can use the power to change configuration attributes to compromise
the security of your Condor pool.
The control lists are defined by configuration settings that contain SETTABLE_ATTRS in their name. The name of the control lists have the following form:
SUBSYS_SETTABLE_ATTRS_PERMISSION-LEVEL
The two parts of this name that can vary are PERMISSION-LEVEL and the SUBSYS. The PERMISSON-LEVEL can be any of the security access levels described earlier in this section. Examples include WRITE, OWNER, and CONFIG.
The SUBSYS is an optional portion of the name.
It can be used to
define separate rules for which configuration attributes can be set
for each kind of Condor daemon (for example, STARTD, SCHEDD, MASTER).
There are many configuration settings that can be defined differently
for each daemon that use this SUBSYS naming convention.
See section 3.3.1 on
page for a list.
If there is no daemon-specific value for a given daemon, Condor will
look for SETTABLE_ATTRS_PERMISSION-LEVEL .
Each control list is defined by a comma-separated list of attribute names which should be allowed to be modified. The lists can contain wildcards characters (`*').
Some examples of valid definitions of control lists with explanations:
SETTABLE_ATTRS_CONFIG = *Grant unlimited access to modify configuration attributes to any request that came from a machine in the CONFIG access level. This was the default behavior before Condor version 6.3.2.
SETTABLE_ATTRS_ADMINISTRATOR = *_DEBUG, MAX_*_LOGGrant access to change any configuration setting that ended with ``_DEBUG'' (for example, STARTD_DEBUG ) and any attribute that matched ``MAX_*_LOG'' (for example, MAX_SCHEDD_LOG ) to any host with ADMINISTRATOR access.
STARTD_SETTABLE_ATTRS_OWNER = HasDataSetAllows any request to modify the HasDataSet attribute that came from a host with OWNER access. By default, OWNER covers any request originating from the local host, plus any machines listed in the ADMINISTRATOR level. Therefore, any Condor job would qualify for OWNER access to the machine where it is running. So, this setting would allow any process running on a given host, including a Condor job, to modify the HasDataSet variable for that host. HasDataSet is not used by Condor, it is an invented attribute included in the STARTD_EXPRS setting in order for this example to make sense.
The condor_ collector and condor_ negotiator daemons use well-known port numbers by default. The condor_ collector uses port 9618 and the condor_ negotiator uses port 9614. All other Condor daemons use dynamically chosen ports. Other port numbers may be chosen by modifications to the configuration.
All of these ports must be greater than 0 and less than 65,536. They ought to be greater than 1024 in order to avoid port restrictions on your machine, in the case that these daemons are not run as root. Also, they need to be ports that are not in use, or the daemons will get very unhappy and pout.
Many more details on port usage are located in section 3.10.8.