Predictive
Network Anomaly Detection and
Visualization
Abstract:
Various approaches have been developed for quantifying and displaying
network traffic information for determining network status and in detecting
anomalies. Although many of these methods are effective, they rely on the
collection of long-term network statistics. Here, we present an approach that
uses short-term observations of network features and their respective time
averaged entropies. Acute changes are localized in network feature space using
adaptive Wiener filtering and auto-regressive moving average modeling. The
color-enhanced datagram is designed to allow a network engineer to quickly
capture and visually comprehend at a glance the statistical characteristics of a
network anomaly. First, average entropy for each feature is calculated for
every second of observation. Then, the resultant short-term measurement is
subjected to first- and second-order
time averaging statistics. These measurements are the basis of a novel
approach to anomaly estimation based on the well-known Fisher linear
discriminate (FLD). Average port, high port, server ports, and peered ports are
some of the network features used for stochastic clustering and filtering. We
empirically determine that these network features obey Gaussian-like
distributions. The proposed algorithm is tested on real-time network traffic
data from Ohio University’s main Internet connection. Experimentation has shown
that the presented FLD-based scheme is accurate in identifying anomalies in network
feature space, in localizing anomalies in network traffic flow, and in helping
network engineers to prevent potential hazards. Furthermore, its performance is
highly effective in providing a colorized visualization chart to network
analysts in the presence of bursty network traffic.
Scope of the project:
Various approaches have
been developed for quant identifying and displaying network traffic information
for determining network status and in detecting anomalies. Although many of these
methods are effective, they rely on the collection of long-term network
statistics. Here, we present an approach that uses short-term observations of
network features and their respective time averaged entropies. Acute changes
are localized in network feature space using adaptive Wiener filtering and
auto-regressive moving average modeling. The color-enhanced datagram is
designed to allow a network engineer to quickly capture and visually comprehend
at a glance the statistical characteristics of a network anomaly.
First, average entropy for each feature is calculated for every second of observation.
Then, the resultant short-term measurement is subjected to first- and
second-order time averaging statistics. These measurements are the basis of a novel
approach to anomaly estimation based on the well-known Fisher linear discriminant (FLD). Average
port, high port, server ports, and peered ports are some of the network
features used for stochastic clustering and filtering. We empirically determine
that these network features obey Gaussian-like distributions. The proposed
algorithm is tested on real-time network traffic data from Ohio University’s
main Internet connection. Experimentation has shown that the presented
FLD-based scheme is accurate in identifying anomalies in network feature space,
in localizing anomalies in network traffic flow, and in helping network
engineers to prevent potential hazards. Furthermore, its performance is highly
effective in providing a colorized visualization chart to network analysts in
the presence of bursty network traffic.
Introduction:
Entropy
is another well-known measure for quantifying the information of network
traffic and has been extensively studied for anomaly detection and prevention
[14], [15]. Significant research has also been devoted to the task of studying
traffic structure and flows in conjunction with visual correlation of network alerts
[16]. Most of the approaches are devised based on the long-term statistics of
network traffic entropy [17]–[20]. One example is the work by Eimann, et al.
[17] which discuses an entropy-based approach to detect network events.
Harrington’s work [19] is similar, but uses cross entropy and second-order distribution
to detect changes in network behavior. Lall, et al. [18] employ the
entropy of traffic distributions to aid in network monitoring, while Gu, et
al. [14] utilize an entropy measure to detect anomalies in network traffic.
More recently, Gianvecchio and Wang [20] introduced an entropy-based
approach to detect the exploitation of covert timing channels in network
traffic among the large amount of regular traffic. In Kim, et al. [21], the
data in packet headers are examined using aggregate analysis of correlation
data and discrete wavelet transforms. Statistical
data
analysis of pattern recognition theory is also applied to the same problem with
varying degrees of success [22]. A supervised statistical pattern recognition
technique is proposed by Fu, et al. [23], which requires the complete
statistics of network load and attack. Wagner and Plattner [24] have discussed a
method based on changes in entropy content for IP addresses and ports but have
not attempted to distinguish normal traffic from abnormal.
Other researchers have taken a variety of
approaches. Thottan and Ji [25] apply signal processing techniques to the problem
of network anomaly detection using statistical data analysis. The IP network
anomaly detection is defined in a single class domain in conjunction with the
types and sources of data available for analysis. They present a method based on
sudden change detection on signals from multiple metrics, each of which has
different statistical properties. In Hajji’s work [26 , the approach undertaken
addresses the problem of change in characteristics of network traffic, and its
relation with anomalies in local area networks. The emphasis is on fast
detection for reducing potential impact of problems on network services’ users
by finite Gaussian mixture traffic model and a baseline of network normal
operation as the asymptotic distribution of the difference between successive
estimates of
multivariate
Gaussian model parameters with mean zero under normal operations, and sudden
jumps in this mean in abnormal conditions. In [27], the researchers introduced
a supervised anomaly detection method by concatenating the –Means clustering
and the ID3 decision tree learning. In their work, -Means clustering is carried
out first on training instances to determine number of distinct clusters,
representing regions of similar instances.
An ID3 decision tree is then trained with the instances
in each -Means cluster so that the anomaly detection can be performed via a
score matrix. Authors of [28] make use of the Tsallis (or nonextensive) entropy
to deal with network traffic anomalies. They have demonstrated the
effectiveness of this measure over the
traditional Shannon entropy-based techniques by detecting more network
anomalies and reducing false negatives. In turn, a flexibility improvement in
the detection process is reported due to the finely tuned sensitivity of the
anomaly detection system as opposed to the conventional entropy measure. Kim
and Reddy [29] consider the time series analysis of different packet header
data and propose simple and efficient mechanisms for collecting and analyzing
aggregated data in real-time.
They demonstrate that their proposed signal series
have higher efficacy in detecting attacks than the analysis of traffic volume
itself. Recently, Androulidakis, et al. [30] have proposed a method of
anomaly detection and classification via opportunistic sampling which also
makes use of port entropy. None of the studies described above provides a
method of predicting an attack before it occurs. In this work, we aim to predict network anomalies. We define an
anomaly as any detected network behavior that is of interest to a network or
security engineer such as worm
outbreaks, botnet command and control traffic, misconfigured network devices,
or DoS attacks.
To this end, we statistically analyze network flow
data and apply
Weiner
filtering to pass normal traffic.
EXISTING SYSTEM:
·
In
these methods are able to identify specific packets which match a known pattern
or originate from a specified location.
·
These
signature-based systems fail to detect unknown anomalies.
·
The
work described in considers the detection of network intrusions in covariance
space using pattern-recognition methods.
·
The
paper also describes a way of detecting network problems using their system.
PROPOSED SYSTEM:
·
This method is used displaying network traffic
information for determining
·
Network status and in detecting anomalies.
·
We present an approach that uses short-term
observations of network features and their respective time averaged entropies.
·
The color-enhanced datagram is designed to allow a
network engineer to quickly capture and visually comprehend at a glance the
statistical characteristics of a network anomaly.
·
The novel approach estimation is based on the well
known fisher linear discriminate.
·
The empirically determine that these network
features obey Gaussian-like distributions.
·
This
approach helps in de termining acute and long-term changes in the network
feature space and presents system status in a visually compact information
graph (called datagram).
·
First,
average entropy for each feature is calculated for every second of observation.
Then, the resultant short-term information measurement is subjected to
·
first-
and second-order time averaging statistics.
·
The
colorized dual-entropy-type datagram visualization is devised to help network
engineers interact with the staggering amounts of network data
·
In
method has been tested under load on real-time network traffic data from Ohio
University’s main Internet connection. Experimentation has shown that the
presented algorithm is able to identify anomalies in selected network features
including average port, high port, server port, and peered port.
SYSTEM
SPECIFICATION:
Hardware CONFIGURATION
Ø Hard disk : 40 GB
Ø RAM : 512mb
Ø Processor : Pentium IV
Ø Monitor : 17’’Color
Monitor
Software CONFIGURATION
Ø Front-End : VS
.NET 2005
Ø Coding Language
: C#
Ø
Operating
System : Windows XP.
Ø
Back End : SQLSERVER 2005
No comments:
Post a Comment