Some Exalogic ZFS Appliance security tips and tricks

Introduction

The ZFS appliance that is internal to Exalogic has been configured specifically for the rack, however while it is "internal" there are still a number of configuration options that should be considered when setting up a machine for production usage.  This blog posting is not an exhaustive list of all the security settings that can be done for a ZFS appliance but does pick off some configuration values that should be thought about whenever the appliance is being setup for use.

User Security

Once an Exalogic rack has been installed by default there will be a single root user of the ZFS array defined.  It is likely that other roles may need to create and manage storage space for their specific accounts.  Handing out the root privileges to other users is not recommended.

The permissions are determined via a three layered system.

• Authorizations
• Configuration items have CRUD (Create, Read, Update, Delete) like actions that can be taken.  
• Roles
• Each role defines a number of authorizations that can be performed by a user with that role
• User
• Defines either a local or remote directory based user that is allowed to authenticate to the ZFS appliance, the roles and hence authorizations will determine which activities the user is able to perform.

In most situations that I have come across the ZFS appliance is administered by the rack administrator so all system level configuration can be performed by one user.  However, there is often a need to be able to provide delegated administration to either an individual share or to all shares in a project.

Consider a scenario where the vDC is to be setup with an account that will host all vServers for Application A, the application may require some shares created to host the binaries and configuration files.  The machine administrator can initially create a project, say called application_a.  Then the role for administrating the project can be created.  To do this click on Configuration --> Users and click on the + symbol beside the Roles to create a new role. 

Create role to administer shares for a specific project

For the authorizations select the scope to be that of the Projects and Shares, then chose the exalogic storage pool and the project that was created earlier.  In this scenario we select all authorizations for all shares so that the user can create multiple shares as needed, although all within the context of the project.  (Click on Add to add the Authorisations selected and then click on add to create the user.) It is possible to only allow specific actions on the project or limit the administration to a single share.

Having created the role we now need to create a user and allocate the role to that user.

Creating a user with restricted permissions

In the example shown above we create a local user that will only have the role to administer the Application A project as limited by the selection of the roles associated with the user. 

Should that user then attempt to make a change to anything other than their project/share the system will respond with the following message.

Error reported when the authorisation has not been granted.

Project/Share Security

Having defined a user with limited access to the ZFS device we now turn our attention to the configuration that provides a level of security to help prevent malicious attacks on an NFS mounted share.  Most of the configuration settings for a share can also be set at the project level, as such we will discuss these first and remember that if necessary the inheritance can be overridden to give an individual share a unique configuration.

• General
• Space Usage
• The quota can be used to prevent any shares in this project from exceeding a set size.  Handy to set to ensure that this project does not use all the available disk space on the device.
• Mountpoint
• Not strictly a security feature but it is good practice to always ensure that the project has a unique mountpoint defined.  By default a share will append the share name onto the project's mountpoint to determine the location in the ZFS appliances directory structure the data for the share.  A format that we use is to have all shares given a mount point of /export/<project name>/<share name>
• Read Only
• Obviously not possible in many cases but certainly at the share level you may wish to have the share setup as Read/Write initially and then change it to be read only so that users cannot accidentally delete the data on it.  (For example a binaries only filesystem.) During upgrades it could be switched back to read/write for the duration of the patching.
• Filesystems - LUNS
• Not directly applicable for Exalogic today but certification to use the iSCSI facility of the ZFS appliance is underway.  At which point then setting the user, group and permissions for LUNs created will be required.
• Protocols
• NFS 
• Share Mode
• Set to None so that by default a client cannot mount the filesystem unless they have specifically been given permission as an exception
• Disable setuid/setgid file creation
• Really down to the usage of the filesystem  - see wikipedia for details on setuid/setgid.
• Prevent clients from mounting subdirectories
• Obviously security related but it will be up to the individual usecase to determine appropriate usage.
• NFS Exceptions
• Having set the share mode to None the usage of NFS Exceptions to allow clients to mount the share is mandatory. There are three mechanisms available to restrict access to a particular host or set of hosts.  Restricting by Host with a fully qualified domain name, by DNS domain or by network. 
  In general I have found the restriction by network to be the most useful but that is partly because DNS domains are often not used when setting up for short term tests.  When using the Network Type specify the "entity" to be a network using the CIDR notion.  So for example, I might want to restrict the share to only vServers in the network range 172.17.1.1 through to 172.17.1.14 in which case the entity should be set to 172.17.1.1/28.  The netmask can be taken down to an individual IP address /32 if only one vServer is allowed to mount the share.
  The access mode set to read/write or read only as is needed for the share usage.
  Root Access indicates if the root user on a client machine would have the root access to files on the share.  In general NFS terminology this is known as root squash.

Example NFS setup

• HTTP, FTP & SFTP
• Leave with share mode of None unless there is a specific need to allow these protocols to access data held on the share.
• Access
• This is a tab that has specific information for a share (other than the ACL Behaviour) so should be set independently for each share.  The Root Directory Access specifies the user/group and the file permissions that will be applied to the share when mounted on the client machine.  If using NFSv4 and hence some sort of shared user repository then the user and group are validated against this store, otherwise you can use values such as nobody:nobody to specify the user:group or enter the UID/GID of the users.  These IDs must map onto a user:group ID in the client machine.   The directory permissions set according to the needs of the application.

• ACL
• Very fine grained access to files and directories is managed via Access Control Lists (ACLs) which describe the permissions granted to specific users or groups.  More detail available from Wikipedia or in the NFSv4 specification (page 50) that is supported by the ZFS appliance.  In general I have found the default settings have been enough for my needs where the world can read the ACLs but only the owner has permission to change/delete them.

Administration Security

The ZFS appliance has many configuration settings  however to lock down the appliance it is possible to turn off a number of the services or re-configure them from the default to minimise risk of intrusion.

• Data Services
• NFS
• On a physical Exaloigc you can specify only to support NFSv4 however on virtual NFSv3 is used by the control stack so must remain a supported version.
• When using NFSv4 it is also necessary to specify the identity domain. (See my earlier posting about setting up LDAP to use for shared authentication.)
• iSCSI - If not used then disable the service.  (As of Exalogic 2.0.6.1.0 iSCSI is only supported for the Solaris Operating System.  In future releases it will also be supported for Linux/virtualised racks.)
• SMB, FTP, HTTP, NDMP, SFTP, TFTP can all be disabled unless specifically needed for some function.  (For example, I quite often use the HTTP service to allow easy access to some media files or to host a yum server.)
• Directory Services
• Generally use either NIS, LDAP or Active Directory for a shared identity store.  Turn off the services you are not using.
• System Settings
• Most of the system settings are useful to have enabled on the rack.  The default settings of having Phone home and Syslog disabled are the best bet.
• Remote Access
•  SSH is almost certain to be required to administer the device via the CLI and using scripted configurations.  However if you setup another user with all necessary permissions then it is possible to change "Permit root login" to deselect this option.  This means that it will no longer be possible to use the root account to ssh onto the rack.  NOTE - If using exaBR, exaPatch, exaChk etc. then these rely on ssh access as root so the flag would need to be toggled back prior to running these tools.

 By default the appliance can be administered on all networks.  This can be tightened up so that administration can only occur over the specific management networks.  To disable administration on a particular interface select the Configuration --> Network --> Configuration tab and then highlight the Interface that you want to disable and click the edit icon to change the properties and deselect the Allow Administration option.

Preventing administration on a particular interface

It is possible to prevent administration on all the networks but the recommendation is to simply prevent it from the networks that a guest vServer can join.  Namely the IPoIB-vserver-shared-storage and the IPoIB-default.  These interfaces can be identified by the IP addresses or partition keys in the description shown in the browser interface.  The IPoIB-default network belonging to "via pffff_ibp1, pffff_ibp0" and the storage network will normally have an ip address in the 172.17.n.n network and be on partition 8005.  (via p8005_ibp1, p8005_ibp0) The partition for the shared storage may vary as it is configurable as part of the Exalogic Configuration Utility on the initial installation.

The effect of deselecting "Allow Administration" on the interface means that a browser will see an "Unable to connect" error and if the ssh interface is used then the following message is shown.

# ssh donald@172.17.0.9Password: Password: Last login: Tue Feb 18 11:51:00 2014 from 138.3.48.238 You cannot administer the appliance via this IP address. Connection to 172.17.0.9 closed.

Summary

In conclusion, there are actually relatively few actions to be taken from the default settings of an Exalogic ZFS appliance but the following should always be considered:-

1. Setup users to administer the projects and shares that are limited to only have write access to the shares they need.
2. For each share make certain that only the protocols that are needed are allowed access (normally NFS only, and potentially iSCSI in the future) and ensure that only specific hosts are allowed to mount the shares
3. Prevent administration on the networks that are connected to guest vServers.

Creating a project/share on the Oracle ZFS Storage Appliance

To quote from a colleague - a tea break snippet (See The Old Toxophilist) on setting up a project and share using the ZFS Storage Appliance that is part of the Exalogic rack.

If you are doing this through the Browser User Interface (BUI) then the first thing to do is point your browser at the management administration interface for the ZFS Storage appliance on port 215. Connecting to the active storage head.

https://<IP of active storage head>:215/

Log on to the service and navigate to the shares tab then pick the Projects sub-tab.

You can then click on the small + symbol beside the Projects title, as shown below.

Give your project a suitable name, say MyProject. You can now select this project from the Projects page to edit it. This is done by highlighting the MyProject line and clicking the pencil icon to edit it. Now we want to do some basic best practice configuration.

1. Click on the General tab and specify the "Mountpoint" to be /export/<project name> This will mean that all the data and shares held in this project will be contained within a single directory structure on the storage device. The rest of the General settings can be left at the defaults in the first place.
   eg. /export/myproject  (Minor unix standard to use lower case characters, if you do use mixed, bear in mind it is case sensitive.)
2. Click onto the Protocol tab.
   
   1. Set the Share Mode to be None. This stops anyone but the nodes that are specifically defined connecting to the share.
   2. Click on the + symbol beside the "NFS Exceptions" to add an exception. I tend to use the Type of "Network" and define a network/netmask as the Entity to specify which compute nodes/vServers can access the share. In a virtualised Exalogic the default vServer shared storage network is 172.17.0.0/16 so giving these read/write access is the norm. There is also a tick box for "Root Access", this defines what is known as root squash which determines if the root user of a connected client has root access to the files in the share. Unless specifically needed this should not be enabled.
   3. Add additional networks as needed for your environment.
   4. HTTP - if you require access to the shares via an HTTP interface then set the share mode for this protocol to be read only.
   5. Replication - No need for this in a very simple test environment but for all other environements the Replication tab allows you to define backup locations for the share.
3. Click on the Shares tab
   1. For each share you wish to create click on the + symbol beside the Filesystems, give your share a name. The other options such as the User and Group and permissions are really dependent on what the needs of your environment are. In the example shown below the assumption is that the share myshare will be mountable from /export/myproject/myshare (the default), and once mounted will show up as being owned by oracle:oracle.
      Note :- you may find that the appliance objects to the owner of oracle:oracle as an unknown user and group. If you are just using NFSv3 then you can enter the ID for the oracle user in here which will transfer over to the client server. If using NFSv4 then the user must exist in the shared authentication location - LDAP or NIS.

Now all you need do is mount this share from a compute node (Physical Exalogic) or vServer (virtual Exalogic)

# mount <ip of storage>:/export/myproject/myshare /mnt

or if you want it to be auto-mounted on boot add it to the /etc/fstab file on a directory such as /u01.