Study and development of an innovative 3G/4G wireless network analysis tool

University of TRENTO - Faculty of Engineering Master of Science in Telecommunications Engineering Study and development of an innovative 3G/4G wireless network analysis tool Advisors Prof. Andrea MASSA Dr. Giacomo OLIVERI Ing. Lorenzo GANDINI (Vodafone) Student Alessandro POLO TRENTO 31 October 2012

Outline Scenario and Goals Mobile networks Key Performance Indexes Detection Methodologies Statistical Wavelet Pattern Matching Filtering Ranking and Classification Filters Conclusions and Future Work EVoKE 2

Scenario 2G, 3G, 4G Wireless Networks Cellular Network Heterogeneous (many vendors, technologies) Growing (services, users) Dynamic (architecture, behavior) Basic Component: CELL Challenges Anomaly Detection Anomaly Classification Vodafone World 238.000 Base Station Sites One Trillion minutes of calls 216 Petabyesof data Vodafone Annual Report 2012 Early 4G Network Architecture How? KPI Analysis 3

Key Performance Indexes Diagnostic Stream KPI Time Series Performance measurement (e.g. number of failed calls: DROPS) Discrete Multivariate random process Mathematical Distribution (PDF) KPIs Thesis Objectives Anomaly Detection Identify anomalies (e.g. peaks/zeros) Alarm Ranking Minimize False-Alarm Rate Live DATABASE Italy 2G, 3G > 1 Gb/day to evaluate CELL 122412 (Duomo, MI), Rate of 3G failed connections Tools Statistical Methods Pattern Matching (Haar Wavelet) 4

Approach KPIs Analysis Live Input: KPIs 1. Detection 2. Filtering Results RAW data Detected Alarms Filtered Alarms 1. Detection CELL 129692 (Medolago, BG), HSDPA Establishment Failure Rate Evaluate RAW data (KPIs) Identify any possible anomaly Generate Alarms 2. Filtering Post-process Detected Alarms Alarms Aggregation, Ranking False-Positive Reduction CELL 142213 (Basaluzzo, AL), Dropped Calls Constraints Data size Processing Time 5

1. Detection a) Statistical Approach IDEA Evaluate sample vs. statistics (range, mean,..) within a Window Available Voice Traffic Channels of CELL 100868(Viale Piave, MI) #Traffic Channels: Low Alarm Example: IF sample<< expected value (e.g. median) Evaluation Window: Length = N previous samples (depends on Detector settings) Anomaly Found Point Anomaly Advantages Simple, Computationally Efficient No training Universal parameters (thresholds) Drawbacks Window s Length (lower-bound) Data Distribution (PDF), Masking Prone to non-zero trend, step Methods Selection: Which statistical tests? 6

1. Detection a) Statistical Methodologies Method Outlier Test Notes Fixed and Adaptive Threshold Three Sigma (*Multi-pass) MAD E 3 Median Rule (x i - reference ) > reference (x i - xˆ) > 3 ˆ σ 0.6745 (x - ~ i x ) MAD (x ~ i - x ) > 2.3 IQR > threshold 3 Processing Robustness, Adaptability Calibration Processing Masking Window Length Robustness MAD Computation Robustness Performance PDF Symmetry(skewness) Robust Estimators use median instead of mean Universal Thresholds easier calibration 7

1. Detection: Examples a) Statistical Detection Good Results: Outlier Detection CELL 100942 (Piazza Lima, MI), Dropped Calls Robust to Noise No Calibration CELL 101033 (Rho, MI), Dropped Calls But still problems: Collective Anomalies Prone to Trends Prone to Steps How to detect patterns? 8

1. Detection: Examples a) Statistical Detection Good Results: Outlier Detection CELL 100942 (Piazza Lima, MI), Dropped Calls Robust to Noise No Calibration CELL 101033 (Rho, MI), Dropped Calls But still problems: Collective Anomalies Prone to Trends Prone to Steps How to detect patterns? 9

1. Detection: Examples a) Statistical Detection Good Results: Outlier Detection CELL 100942 (Piazza Lima, MI), Dropped Calls Robust to Noise No Calibration CELL 101033 (Rho, MI), Dropped Calls But still problems: Collective Anomalies step Prone to Trends Prone to Steps How to detect patterns? 10

1. Detection b) Wavelet Comparer IDEA Pattern Matching in Wavelet Domain Why? Wavelets Multi-Resolution Time and Frequency Localization CELL 134801 (Lainate Sud, MI) Ramp or Trend Which Mother Wavelet? Haar Wavelet Advantages Computational Complexity O(N) Easy (local) Trend recognition CELL 100492 (Basiglio, MI) Step Drawbacks Based on mean, not robust Prone to punctual outliers How to detect patterns? 11

CELL 101461 (Meda, MI), Dropped Calls 1. Detection b) Wavelet Comparer Haar Decomposition Example DROPS Does input signal match given pattern? Signal = { 9,7,3,5 } DWT = { 6, 2, 1, -1 } Input Signal Ramp Pattern Approximation 2 (mean) Time Domain Apply HAAR DWT (Level 2) Haar Wavelet Domain Details 2 Higher Resolution Higher Frequency Sign Matching Trend Matching Details 1 Approx 2 D 2 D 1 1 D 1 2 Pattern Found D 1 1 D 1 2 12

1. Detection b) Wavelet Comparer: Example CELL 101581 (Gattinara Centro, VC), HSDPA Establishment Failure Rate HSDPA EFR Monotonic Step Pattern Input Signal Mismatching Sign Pattern NOT Matching Haar DWT Coefficients Approx 3 D 3 D 2 1 D 2 2 D 1 1,2,3,4 13

2. Filtering Challenges Live Input: KPIs 1. Detection 2. Filtering Results RAW data Detected Alarms Filtered Alarms Filtering Objectives Alarm Ranking Aggregation Classification False-Positives Reduction Reduce work load Network Maintenance Efficiency Highlight Alarms of Interest Automate Best Practices Integrate Know-How Network operators are busy, the system is designed to improve efficiency How? 14

2. Filtering Intrinsic Detection Ranking Alarm Priority? Deviation from Expected value High Score! Score event i = reference - x reference (e.g. reference = median) CELL 131881 (Arluno Casello A4, MI), Dropped Calls i Is this enough for domain experts? No, because: Priority is Affected by Previous Behavior KPI, Cell relevance Not static(change in time).. Lower Score but the Alarm is persistent, It s priority Domain Experts require Smarter Ranking 15

2. Filtering Improve Ranking: Example IDEA Tune Score by means of ad-hoc Filters derived by: Domain Experts CELL 131881 (Arluno Casello A4, MI), Dropped Calls Persistent Alarm Filter Filtering Increased Score 16

2. Filtering IDEA Tune Score by means of ad-hoc Filters derived by: Domain Experts Holiday Alarm Low Network Load Recognize Holidays Dynamic Vacation DB Reduce Ranking CELL 110907 (Calcinate,BG) Traffic [Erlangs] Traffic is lower during vacations Lower Score Persistent Alarm CELL 131881 (Arluno Casello A4, MI), Dropped Calls Higher Score Evaluate History Time Correlation Increase Ranking Link Alarms Recurring Alarm Isolated Alarm Future Observations Time Correlation Reduce Ranking Reduce False-Positives CELL 100069 (Via Napo Torriani, MI), Dropped Calls Single Spike, False-Alarm? Lower Score 17

Summary Results 1. Anomalies Detection (green samples) 2. Aggregation, Tagging, Re-Scoring (colored labels) CELL 113433 (Paderno-castelletto, MI), Dropped Calls Refined Score = 52 Refined Score = 87 (even if sample is lower) Decreased Score Increased Score False-Positives Less priority Alarm Notified More priority Comparison with State of the Art Commercial tool 18

Comparative Assessment (vs. ANTARES) CELL 113433 (Paderno-castelletto, MI), Dropped Calls, Analysis by Vodafone ANTARES Vodafone ANTARES State of the Art Commercial tool 45 alarms Many false alarms! -64% False Alarms 19

EVoKE Demo Analysis Tool Single Day 28 August2012 North-West Italy Network: 2G, 3G Nokia (31571 cells) 4 KPIs 2G (GSM) 2 KPIs 3G (HSDPA) 4 Detectors 5 Filters Processing Time 16 seconds 20

EVoKE Geo Viewer Visual Navigation Shown Example Daily Results 28 August2012 North-West Italy Network: 3G Nokia (18087 cells) KPIs: 3G HSDPA Traffic [Kb] Establishment Failure Rate [%] Spatial Correlation Many alarms, N N Many alarms, Few nearby alarms, Low resolution Higher resolution N Highest Resolution 21

Conclusions Resume Scenario: Mobile networks, KPIs Detection: Statistical Methods, Haar Wavelet Filtering: Ranking and Classification Results: EVoKE Future Work Alarms Correlation(Space, Causality) Classification (SVM, Bayesian networks) 22

Thank you 23