IDS and IPS in network architecture
A firewall controls traffic at a zone boundary, but its decisions use the fields, state, and rules visible to it. Allowed traffic can contain an application attack, and a threat can originate on an internal system. Security monitoring therefore also covers communication within the network, host state, and user-related events. The observation point determines which devices and directions are visible, while log retention determines how long an alert can be checked. Design should define the required scope, storage cost, privacy controls, and incident-response owner. It should also set escalation time and the required audit format, including the records needed for later review.
and Intrusion Prevention Systems (IPS) observe network traffic or host events. An IDS creates an alert, while an IPS can run inline and block matched traffic. Both require data sources, rules, updates, alert handling, and an assessment of service availability. An alert should include time, source, target, rule, confidence, and action so the assessment can be repeated.
Intrusion Detection System (IDS)
An monitors data and creates alerts for matched signatures, anomalies, or policy violations. In a typical deployment it operates outside the traffic path and does not block the event. Other configurations can connect an alert to a separate response mechanism, so the system design must define the actual behavior. The alert pipeline should specify deduplication, severity mapping, notification, and retention for the original observation.
An IDS may receive a copy of traffic from a SPAN port, mirror port, or packet broker. Its analysis engine compares data with rules, records the result, and forwards an alert to event management. The alert describes an observation and a rule match. An administrator decides whether the event requires host isolation, additional evidence, or a rule change. The design should account for time synchronization, log protection, mirror-interface load, and handling of missing packets.
An out-of-band deployment separates an IDS failure or delay from the main packet-forwarding path. The IDS cannot stop traffic, however, and a mirror port may miss frames when overloaded, when the VLAN scope is wrong, or when encrypted payloads are not visible. The design should state which points are monitored and how missing data is handled. Visibility should be tested after topology changes, and input-frame counts should be compared with the number of frames analyzed by the IDS.
IDS types by observation point
NIDS observes communication in a segment, while HIDS collects events from one host. The types can complement each other during an investigation, but their data sources and limits differ.
1. Network-based Intrusion Detection System (NIDS)
A NIDS is placed where it can receive a copy of traffic from a segment. Depending on topology, mirror scope, and filters, it may observe communication among servers, workstations, and the Internet. The sensor location should be documented with links, VLANs, expected directions, and a method for checking that the copy is complete.
A NIDS can flag a scanning pattern, such as a host at attempting connections to multiple servers and ports. The alert depends on traffic visibility and rule thresholds. The address is an observed endpoint, so its owner must be established from separate logs. With NAT, the analysis should include source port and time; traffic between switches also requires the correct mirror scope.
A NIDS usually cannot see content protected by HTTPS or another TLS channel. It can still analyze metadata, handshake parameters, addresses, sizes, and frequency. Full content analysis requires a view after decryption or controlled TLS inspection.
2. Host-based Intrusion Detection System (HIDS)
A HIDS is an agent on a specific host, such as a web server, database server, or laptop. It collects system events, processes, file changes, authentication logs, and selected traffic data processed by the host. The policy should state which events are mandatory, how the agent is protected, and where its records are stored.
A HIDS watches for file changes insidethe host, such as a modification of '/etc/passwd', process creation, a registry change, or an authentication event. Scope and alert reliability depend on agent privileges and protection of its logs.
A HIDS can detect a file change or process launch after an infected removable drive is connected, even when a NIDS did not receive the relevant traffic. The agent can also analyze traffic after decryption on the host. It does not replace a NIDS because an agent failure, evasion, or missing privilege limits observation. Deployment requires protection against agent shutdown, performance checks, and forwarding logs to an independent store. For critical hosts, the team should define mandatory events and a method for confirming that the agent is operating.
Detection methods
An IDS compares observations with knowledge of known events or with a model of ordinary activity. Both methods require tuning, updates, and alert handling.
1. Signature-based detection
A signature describes a pattern associated with a known threat, such as a byte sequence, a SQL injection command, or a characteristic scan. The engine compares observed traffic and host events with a rule library. Results depend on visibility, protocol normalization, signature versions, and update procedures. Rules can include priority, context conditions, and exceptions that need testing after application changes.
A signature match shows similarity between an observation and a known pattern. It does not identify an undescribed variant, prove impact, or replace checking the application context.
Strength: A tuned rule can quickly identify known patterns and provide a detailed event description.
Weakness: A new variant or zero-day threat may have no matching signature. Effectiveness depends on updates, rule quality, and exceptions. Broad matches increase false alerts.
2. Anomaly-based detection
An anomaly model describes typical activity by protocol, server relationship, time of day, packet size, and transfer volume. An alert is produced when a deviation crosses a configured threshold. The model may need a learning period and regular adjustment as the network changes.
An illustrative anomaly is a host that transfers gigabytes of data at 3 AM even though it rarely contacts an external server. The pattern justifies checking the backup schedule, host role, and application logs. It is not by itself proof of an attack because a legitimate administrative task can look similar.
Strength: It can identify a new pattern that is absent from a signature library, including a possible zero-day attack.
Weakness: Unusual but legitimate activity can generate an alert. A rare diagnostic tool, a schedule change, or a one-time large transfer can increase noise. The team must set thresholds, exceptions, and an assessment process.
Intrusion Prevention System (IPS)
An analyzes traffic and can execute an action after a rule match. Unlike a typical IDS, it is deployed inline or connected to a mechanism that can block, reset, or limit communication.
In inline mode, packets pass through the IPS before reaching their destination. The device can use signatures, anomaly analysis, and protocol rules. After a match it can drop a packet, block a flow, reset a session, or forward an alert. Availability depends on throughput, exceptions, and fail-open or fail-closed configuration.
Inline deployment creates a dependency on the device. Failure, overload, a wrong signature, or an overly broad rule can add latency, block legitimate traffic, or interrupt a service. The design needs bypass, monitoring, limits, and failure handling. Before blocking is enabled, the team should test monitor mode, rollback procedures, and effects on long-lived connections. Tests should cover encrypted traffic, fragmentation, different protocols, and production-like load.
IPS response actions
After a rule match, the IPS can take an action specified by policy:
- Drop packets:The IPS discards packets that match a rule and records the decision. The effect depends on the protocol and the device's position in the path.
- Block traffic: The IPS can add a time-limited rule for an address, port, or flow. An automatic block needs a timeout, exceptions, and review to limit the effect of a wrong match.
- Reset the connection: For TCP sessions, the IPS can send RST segments to the parties and interrupt the connection. This requires visibility in both directions and can be limited by topology or by traffic that is not visible at the enforcement point.
- Alert and log: The IPS records the match, decision, and relevant traffic fields and forwards an alert for review according to its configuration.
Alert classification errors
IDS and IPS can classify an observation incorrectly. Analysis distinguishes legitimate activity marked as a threat from an attack that the system missed. Both errors affect operations and service availability.
False positive
A false positive occurs when the system marks legitimate activity as malicious.
Many false positives create alert fatigue and delay the review of important events. In an IPS, a wrong decision can block a user, a critical application, or a partner connection. Rule tuning should account for exceptions, deployment mode, service priority, and the ability to withdraw a block.
False negative
A false negative occurs when the system fails to detect an actual attack or fails to forward the appropriate alert.
Causes include missing visibility, encryption, an outdated signature, rule evasion, overload, or an event assigned too little priority. The absence of an alert does not establish the absence of activity, so analysis should include host data and sources independent of the IDS.
Tuning an IDS or IPS involves measuring both error types, updating rules, reviewing exceptions, and checking the effect of blocking on services. Sensitivity, reaction delay, and availability are set according to risk and system role. Results should be reviewed after signature updates, topology changes, and the deployment of a new service. A review record should identify the tested traffic, changed thresholds, observed errors, decision owner, and follow-up date.
Integrated security functions
Security products can combine filtering, application identification, IDS, IPS, VPN, and endpoint protection. A product name does not define the exact feature set, so the operating mode, signature sources, and configuration must be checked.
- Next-generation firewall (NGFW): An NGFW typically combines stateful filtering with application identification, deep packet inspection, and selected IPS functions. Full content analysis depends on encryption, the module, and inline mode. Signatures, blocks, and exceptions require separate configuration.
- Unified threat management (UTM): A UTM can combine a firewall, NIDS/NIPS, gateway antivirus, content filtering, and VPN in one appliance. Features depend on the product, license, and capacity. Consolidation can simplify management, but throughput, visibility, and failure effects need assessment.
An IDS provides observation and alerts, while an IPS can execute an inline response. A useful architecture combines these functions with host logs, incident management, and change control. Protection depends on visibility, rule quality, updates, tuning, and testing the effect on services. The design should also name the alert owner, response time, escalation path, and required audit record.