Cascade Strategies HIPAA-compliant
System Security Provisions

Overview of System Availability

• Multiple ISP connections to the data center provides uninterrupted high-speed internet access at all times.
• Network infrastructure configured in high availability with N+1 firewalls, switches, load balancers and server NICs.
• Fault-tolerant data storage: Microsoft SQL Server instances run on 2-node SQL Server clusters with Storage Area Network (SAN) storage dedicated to each instance. Survey databases are distributed across instances on multiple clusters.
• Application and web servers are deployed in load balanced mode with several back-end servers available to provide N+1 redundancy and seamless updates.
• Dedicated VMware virtual infrastructure (private cloud) providing at least N+1 fault-tolerance for both virtual servers and hypervisors.
• Multi-path fibre channel network for SAN connectivity and EMC VNX2 storage arrays for all storage.

Security

Organizational Controls:

• Our system includes several security controls, built into the system, including:

o       Granular access controls limiting user access, even within the same client account.

o       Hierarchical role and permission assignment using groups to simplify enforcement.

o       Strong Password requirements, including validation against PWNED database.

o       Automatic account lockout caused by failed login attempts.

o       Passwords are stored in one-way salted and hashed format.

o       2FA that users can enable on the account.

o       Automatic session timeouts.

• We commission independent third-party security specialists to run application testing of our software. The tests are run annually.

o       Application testing: We grant a user a valid password and User ID to our software, and see if they can “hack” any part of the system, i.e. gain illegitimate access to data, elevate permissions, compromise the software, etc.

o       Any relevant findings are promptly corrected, and retesting is carried out to verify fixes.

• Weekly static code-scanning of our software, which is:

o       Integrated into our Software Development Life Cycle.

o       Performed by industry leading static code analyzers.

o       Automated scans ensure systematic detection and reporting.

o       Frequent: weekly scanning for rapid identification and remediation.

• 24x7x365 Security Operations Center (CSOC) staffed by GCIA- and GCIH-certified security analysts who triage and investigate all alerts as well as well as perform active hunting missions for threats and indicators of compromise that may not be detectable by tooling.
• Host protection capable of prevention and detection of threats.
• SIEM capabilities for anomaly detection on the infrastructure and alerting to the 24×7 CSOC.
• Rapid-response remediation – The CSOC is authorized to perform pre-approved actions and remediation 24 hours a day.

Network/System Security:

• Threat Management systems will automatically raise alerts related to malicious network traffic flowing into to the environment. Alerts are monitored and handled, and offending addresses/networks can be blocked on the firewall level.
• Remote administrative access is controlled and only available through secure VPN channels requiring 2FA (two-factor-authentication).
• Firewall rule sets designed after ‘deny all, allow only required services’ principle.
• Dual firewall layer design with network segmentation providing additional security for servers storing data.
• HTTPS Everywhere in force: All application logins are enforced to encrypted TLS/HTTPS connections.
• Data encryption in effect for data at rest, for exports, and in transit (SFTP, TLS, PGP).
• DDOS protection to limit the impact of distributed network-based volumetric attacks.
• Security alerts are sent to us, in addition to creating a security incident ticket.

Server Configuration / Firewalls

Site Configuration

In order to ensure the best possible availability, all components are deployed an N+1 high availability configuration, from Internet connections and firewalls to servers. All servers are connected to redundant Gigabit (or better) network switches using a teamed NIC configuration. A pair of load balancers, also configured in active/passive failover mode are connected to the switches. The load balancers distribute traffic designated for various web servers based on least connection algorithms for optimal performance.

The load balancers, which are upgraded with fully functioning stateful firewall feature sets, provide network segmentation for different server groups security zones (Web/Utility/DMZ/DB/).

All servers are located on an internal subnet using NAT between the private and public interfaces on the firewalls, with network zoning configured on firewalls to separate public from private traffic. A second pair of firewalls is used to further segregate access to servers storing data. All servers within the site are members of a dedicated Active Directory domain, contained within their own OU, ensuring that only authorized users and administrators can view and access the servers. Group policies are used to control server and user/group security.

We monitor and perform administration from the management network. Only specific IP address ranges from the Rackspace management networks are allowed through the firewalls for monitoring the devices. Only these same IP addresses will have access to the Rackspace data center infrastructure (switches/routers) as well.

Server Configuration

We use a kick-start network boot system, which will ensure that all servers are built to the same and updated specification. Post-install configuration and server hardening is performed by our operations team utilizing security benchmarks from Center for Internet Security.

Application and web servers run on a dedicated VMware vSphere infrastructure. The virtual platform is fully managed by us but is dedicated to running our servers.  Virtual servers are cloned from a default server template that has already been pre-configured and hardened, allowing for rapid deployment based on demand.

Firewall Setup and Security

The platform is protected by two sets of redundant firewall devices, each set from a different manufacturer.  All network equipment is configured in high-availability mode spanning two separate racks with individual power supply.

Firewalls are configured in an active/backup high availability configuration. In this configuration, one of the firewalls acts as a primary device and will handle all traffic to and from the site on a day to day basis. In the event of a failure on the primary device, the backup firewall will automatically take over and begin handling the network traffic, transparent to the end user.

Servers are connected to different network segments according to their role. Servers storing data are located behind an internal pair of firewalls that only allow specific traffic from known servers in the segments behind the external firewalls and allow no direct traffic from Internet sources.

Firewalls have real-time logging and alerting capabilities, and their operation is monitored 24×7. The firewall rule set is based on our internal management and monitoring infrastructure.

Firewall rule set configuration

The firewall rules are designed with a default ‘deny all’ rule that applies to all services that are not specifically allowed through rule sets.  Rules created for our system include:

• Port 80 (HTTP) and port 443 (HTTPS) are required for accessing web applications and web services. These ports are opened inbound to the load balanced IP addresses on the load balancer, which interprets and forwards traffic to the relevant servers that comprise the service behind the load balanced IP. All traffic on port 80 is redirected to port 443, and HSTS is enabled to ensure that traffic stays on HTTPS.
• Port 22 (SSH/SFTP) is required for data up- and downloading using the file transfer add-on. The service is available for authorized users only and is opened only for the server that runs this service.
• Port 25 is required for allowing remote mail servers to connect to outbound mail servers to verify that mail servers can be contacted for deliveries of bounces/NDRs (some remote servers perform a remote connection to a mail server to verify its existence before they accept mails from the originating server).

Email servers are configured to reject external relaying.  Our email servers only perform outbound mailing actions and do not accept remote delivery.

Internal firewall rules for traffic flowing between different network segments in the environment are configured only to allow required traffic towards application groups that are required for specific server roles.

Threat Management

A Threat Management System is installed on the network. This system constantly inspects traffic packets and checks against known patterns based on signatures from various attacks and malicious requests. Signatures are continuously updated through a centralized system, and new signatures are added frequently, based on new traffic patterns learnt from devices deployed in multiple regions around the globe. Some devices are deployed in highly volatile networks and will often allow the Threat Management System to recognize the pattern of a new type of attack and provide a signature for it even before it is released ‘in the wild’.

Penetration Testing

External penetration testing is performed by on an annual basis to identify potential vulnerabilities in internet facing services.

System Management and Monitoring

Our software runs on a standardized hardware platform that allows for easy management and maintenance, part replacement, troubleshooting, monitoring, and capacity planning.  We perform real-time monitoring and alerting.

Capacity requirements are monitored, trended, and regularly reviewed for a variety of factors such as CPU/Memory pressure, disk space, backup disk space, bandwidth, disk IO, database server health, application response time, etc.

All systems are monitored 24/7/365 using local, centrally managed, and external tools. Alert triggers are configured to notify Cloud Operations on-duty personnel in case of any malfunctioning system.

Locally managed monitoring tools

A Performance Monitor is deployed on selected servers to gather data about processor utilization, memory utilization, disk I/O statistics, and database statistics.

Centrally managed monitoring tools

Our centrally managed monitoring tools provide a variety of sensor types, supports custom alerts, and our personnel will be notified immediately in the event of an application component anomaly.

Additionally, we have developed customized monitoring tools in-house for specific monitoring of key performance indicators pertaining to our software.  We also deploy database server monitoring (currently using Spotlight on SQL Server). We use Grafana tools for monitoring Kubernetes workloads.

We provide the following centrally managed monitoring tools:

• Microsoft System Center Operations Manager– SCOM – agents are deployed on applicable servers. The agents monitor processor, memory, disk, and system process/services health as well as hardware health as reported from server vendor agents.
• Web application availability and performance monitoring.
• Bandwidth usage and network latency.
• VMware Vcenter is used to monitor VMware platform performance.
• All reports are online available to us, in addition reports are reviewed monthly alongside the dedicated account team.

Information available to us through the Performance Monitoring systems are:

• System Profiles
• Performance Profiles

o     CPU Utilization
o     Disk Utilization
o     Memory Utilization

• Firewall rule set design
• DNS management
• Network Performance
• Network Utilization
• Backbone Status
• Backup Status
• Bandwidth Status
• Security

External monitoring tools

We use Site24x7 (https://www.site24x7.com) monitoring to verify web application availability and performance from multiple locations.

• Authoring access is being monitored using a ‘real browser’ transaction performing a full application login with content check.
• Reportal access is being monitored using a ‘real browser’ transaction performing a full application login with content check.
• Survey access is being monitored using a ‘real browser’ transaction by completing a scripted survey that performs data submission and redirect.
• Alarms are configured to trigger within various performance and availability thresholds and will notify our operations team upon exceeding configured limits.

Logging and reporting

OS-level Logging and monitoring

OS-level and web server logging has been enabled on all our servers. The Security Audit Event log is configured via domain policy.

The following table illustrates the logging performed by the Security Audit:

We examine logs whenever a server incident is logged, or when a support request is raised requiring log analysis. Logs are archived and stored off-site for at least 12 months.

Logging and monitoring

Several activities within our system are being logged. A fully updated overview of the logged events can be found in the User Documentation. Application access and service logs are parsed and indexed on an ELK (Elastic, LogStash, and Kibana) cluster for searchability and building performance/usage/health trend dashboards.

Incident Management

Our Incident Management procedure is invoked for any serious security incident for our system. The procedure will also be triggered by malicious use or access, virus attacks, Trojans, DOS attacks, spoofing, defacing, fragment attacks or any other hacking or security related incident.  The system will alert us of security alarms triggered by the threat management system or the firewalls. It is our responsibility to follow up and act on the information given by these alerts.

Documentation

• All server set-up and configuration are documented.
• Change control procedures for servers and infrastructure are in place and documented.
• Operations procedures are in place and documented.
• Security processes and procedures are in place and documented.
• Incident management and escalation routines are in place and documented.
• Information Security policy is in place.

Server Security

All servers are hardened to prevent unauthorized access and malicious use.  Recommendations from CIS are used as the baseline.  Hardening steps include:

• All unnecessary system services are stopped and disabled.
• Only required service roles/features are installed / enabled.
• All unnecessary network protocols and services are disabled / blocked in local firewall rule sets.
• Permissions are applied to registry keys and file system to prevent unauthorized access.
• A global domain policy is applied to all servers to ensure identical auditing, account and password requirement settings, and other security policies.
• IIS (Web server components) also undergoes hardening:
• Only required IIS Services components have been installed.
• Web content is moved away from the system drive to prevent directory traversal exploits.
• IIS is hardened using Request Filtering rules.

All servers are updated with latest OS service packs and security updates.  Software installed on servers is fully licensed.  At all times, production software is kept up to date with latest version where applicable. All servers have monitoring and anti-virus software installed.

All users of the systems have a unique user account that will be created on a need-to-access server basis.

• All accounts are unique and regularly reviewed.
• Default system accounts are renamed and disabled.
• Unnecessary user accounts are deleted.
• All accounts have minimum privileges required.
• User activity will be logged, stored, and reviewed.
• A specific local administrator account (for DC Ops emergency use only) is restricted to local console login (remote login prohibited by GPO policy).
• Remote access traverses a bastion server for authentication and auditing. Authentication tokens are provided on a per-request, time-restricted basis.

Antivirus and Endpoint Protection

Antivirus software is deployed on all our servers.  All production servers also have agents installed which report back unusual behavior and activity on servers. All detected anomalies are escalated to the 24×7 CSOC and promptly investigated by our analysts.

Data Transmission Security

Our system uses certificates for web-facing servers that can provide a safe and secure method of accessing the SaaS site for both Project Managers and respondents should the secure link be used. Encrypted traffic is terminated in the load balancers, offloading resources from virtual machines for increased performance.

Backup and Recovery Policy

We have deployed an enterprise backup solution designed for our data center environment.  Backup is performed in two steps: application level backup and offsite backup. The process is outlined below:

Application level backup

Step one is to back up our data including generated survey files and SQL databases. The backup process is outlined below:

Offsite Backup

Step two is to back up the encrypted archives from each server to secure off-site storage locations.  The backups are taken off the backup folders each day.  The weekly backup cycle consists of the weekly full backup and daily differential backups, i.e., the data that has changed since the last backup whether incremental or full.

The minimum on-site retention period is up to one (1) full backup and all differential backups since last full.  The off-site retention of every backup is 52 weeks.

Data Restore

Our employees may request data restore through the support portal if the requested data is no longer contained on the local backup server.  Data will be restored either to the original database, or to a new copy/duplicate of the original database for merging of records.

Physical Environment

Location

Our building has fully redundant connectivity, power and HVAC to avoid any single point of failure and are staffed 24/7 by highly trained technical support personnel.

All critical systems are N+1 resilient to provide uninterrupted availability. All data center systems are tested regularly, including High Availability (HA) testing to ensure continued operation in the event of failures of Power Distribution Units, NIC teams, cluster resources, SAN fibre connections, redundant switches, servers, power supplies, load balancers and firewalls. Weekly tests are conducted on all HVAC, UPS, fire suppression, and generator systems.  There are no windows with access to any secured perimeters in which the servers or shared communication equipment reside.

Site Security

External Access

The data center is equipped with both interior and exterior closed-circuit television as well as physical entry barriers.  Onsite security personnel monitor the data center buildings 24 hours per day, seven days per week. The security team is responsible for making sure that only authorized personnel are allowed into the sensitive areas of the data center building. The security personnel provide the first layer of security for entering the data center.

Internal Access

Multiple levels of security are deployed to ensure that only certified operations engineers are physically allowed into the data center.

• Public access is strictly prohibited. We do not co-locate equipment, which eliminates the need for anyone but our technical engineers to be allowed into the data center facility.
• All employees are given a thorough background check before being hired. Additionally, all personnel go through a rigorous training process before they begin interacting with the systems.
• Live CCTV surveillance of the entire data center building is monitored 24 hours per day. The CCTV recordings are kept on file for at least 90 days. All entrances to the buildings as well as entry and exit points from data center is monitored to ensure that only authorized personnel are allowed into controlled areas.
• Access to the data center buildings is restricted to those who hold a pass card. Once on a specific floor, the pass cards are required for moving from room to room.  The security pass card system represents the second layer of security for entering the data center.
• Biometric hand scanners are used to restrict access to the data center. Biometric security systems represent the third layer of security for entering the data centers.

Visitors

The following rules apply to visitors to the data center:

• Visitors must be signed in and are required to show photo ID upon entering the facility.
• All visitors must wear a badge during the stay and turn it in upon departure.
• Visitors are accompanied by data center staff throughout the visit.
• Visitors can only view the data center through a designated viewing room separated from the server floor by a security glass window.

Environment control

The data center has temperature and humidity control systems.

Disaster Protection – Fire

Fire suppression is achieved through state-of-the-art smoke detection and alarm systems with a zoned, pre-action dry pipe suppression system.

Disaster Protection – Earthquakes

The data center adheres to the standard earthquake local laws and ordinances.

Power

The data center gets power from commercial utility underground conduits with a 30-minute battery backup in the event of failure.  Diesel generators with full-load capability and 24-hour fuel supply are on standby to provide long- term power in the event of an emergency.

The power systems and generator systems are regularly tested to ensure that they will function properly in the event of a power system failure.

UPS Systems

The power systems are designed to run uninterrupted even in the unlikely event of a total power outage. All staging and production systems in the hosting environment are fed with conditioned UPS power that will run if utility power fails. The UPS power subsystem is N+1 redundant with instantaneous failover in case the primary UPS fails.

Diesel Generator Systems

The onsite diesel generators will automatically start in the event of a power surge or power system failure. The power subsystems are designed to cut over immediately with no interruption in the event of a power failure.

Network Design

The data center is connected to multiple ISP backbones to ensure that data reaches the intended destination in the fastest, most efficient manner possible.

The data center backbone runs the Border Gateway Protocol for best-case routing. All internal network equipment has full redundancy (N+1 hot failover) to ensure that data can be routed even in the event of a single switch failure. The BGP protocol standard allows for routing packets of information sent out from the network. Each packet of information is evaluated and sent over the most efficient route available. Because of the redundant network architecture, packets may be sent via alternative routes, even if they are being delivered to the same end user.  Should one of the providers fail, packets leaving the network are automatically redirected through another route via a different provider.

The network currently has excess capacity, even during peak hours. Should peak usage exceed 40% of total capacity, the connectivity will be upgraded.

All wiring trays/cable runs are either in locked cabinets, or ceiling mounted. Only certified individuals are permitted access to these areas.

Cleaning

Regular cleaning within the data center is performed by operations staff. Access to the data center is strictly controlled and limited to employees and trusted maintenance personnel under supervision.

Last Revised: April 10, 2024