Study and development of an innovative 3G/4G wireless network analysis tool

Transcription

1 University of TRENTO - Faculty of Engineering Master of Science in Telecommunications Engineering Study and development of an innovative 3G/4G wireless network analysis tool Advisors Prof. Andrea MASSA Dr. Giacomo OLIVERI Ing. Lorenzo GANDINI (Vodafone) Student Alessandro POLO TRENTO 31 October 2012

2 Outline Scenario and Goals Mobile networks Key Performance Indexes Detection Methodologies Statistical Wavelet Pattern Matching Filtering Ranking and Classification Filters Conclusions and Future Work EVoKE 2

3 Scenario 2G, 3G, 4G Wireless Networks Cellular Network Heterogeneous (many vendors, technologies) Growing (services, users) Dynamic (architecture, behavior) Basic Component: CELL Challenges Anomaly Detection Anomaly Classification Vodafone World Base Station Sites One Trillion minutes of calls 216 Petabyesof data Vodafone Annual Report 2012 Early 4G Network Architecture How? KPI Analysis 3

4 Key Performance Indexes Diagnostic Stream KPI Time Series Performance measurement (e.g. number of failed calls: DROPS) Discrete Multivariate random process Mathematical Distribution (PDF) KPIs Thesis Objectives Anomaly Detection Identify anomalies (e.g. peaks/zeros) Alarm Ranking Minimize False-Alarm Rate Live DATABASE Italy 2G, 3G > 1 Gb/day to evaluate CELL (Duomo, MI), Rate of 3G failed connections Tools Statistical Methods Pattern Matching (Haar Wavelet) 4

5 Approach KPIs Analysis Live Input: KPIs 1. Detection 2. Filtering Results RAW data Detected Alarms Filtered Alarms 1. Detection CELL (Medolago, BG), HSDPA Establishment Failure Rate Evaluate RAW data (KPIs) Identify any possible anomaly Generate Alarms 2. Filtering Post-process Detected Alarms Alarms Aggregation, Ranking False-Positive Reduction CELL (Basaluzzo, AL), Dropped Calls Constraints Data size Processing Time 5

6 1. Detection a) Statistical Approach IDEA Evaluate sample vs. statistics (range, mean,..) within a Window Available Voice Traffic Channels of CELL (Viale Piave, MI) #Traffic Channels: Low Alarm Example: IF sample<< expected value (e.g. median) Evaluation Window: Length = N previous samples (depends on Detector settings) Anomaly Found Point Anomaly Advantages Simple, Computationally Efficient No training Universal parameters (thresholds) Drawbacks Window s Length (lower-bound) Data Distribution (PDF), Masking Prone to non-zero trend, step Methods Selection: Which statistical tests? 6

7 1. Detection a) Statistical Methodologies Method Outlier Test Notes Fixed and Adaptive Threshold Three Sigma (*Multi-pass) MAD E 3 Median Rule (x i - reference ) > reference (x i - xˆ) > 3 ˆ σ (x - ~ i x ) MAD (x ~ i - x ) > 2.3 IQR > threshold 3 Processing Robustness, Adaptability Calibration Processing Masking Window Length Robustness MAD Computation Robustness Performance PDF Symmetry(skewness) Robust Estimators use median instead of mean Universal Thresholds easier calibration 7

8 1. Detection: Examples a) Statistical Detection Good Results: Outlier Detection CELL (Piazza Lima, MI), Dropped Calls Robust to Noise No Calibration CELL (Rho, MI), Dropped Calls But still problems: Collective Anomalies Prone to Trends Prone to Steps How to detect patterns? 8

9 1. Detection: Examples a) Statistical Detection Good Results: Outlier Detection CELL (Piazza Lima, MI), Dropped Calls Robust to Noise No Calibration CELL (Rho, MI), Dropped Calls But still problems: Collective Anomalies Prone to Trends Prone to Steps How to detect patterns? 9

10 1. Detection: Examples a) Statistical Detection Good Results: Outlier Detection CELL (Piazza Lima, MI), Dropped Calls Robust to Noise No Calibration CELL (Rho, MI), Dropped Calls But still problems: Collective Anomalies step Prone to Trends Prone to Steps How to detect patterns? 10

11 1. Detection b) Wavelet Comparer IDEA Pattern Matching in Wavelet Domain Why? Wavelets Multi-Resolution Time and Frequency Localization CELL (Lainate Sud, MI) Ramp or Trend Which Mother Wavelet? Haar Wavelet Advantages Computational Complexity O(N) Easy (local) Trend recognition CELL (Basiglio, MI) Step Drawbacks Based on mean, not robust Prone to punctual outliers How to detect patterns? 11

12 CELL (Meda, MI), Dropped Calls 1. Detection b) Wavelet Comparer Haar Decomposition Example DROPS Does input signal match given pattern? Signal = { 9,7,3,5 } DWT = { 6, 2, 1, -1 } Input Signal Ramp Pattern Approximation 2 (mean) Time Domain Apply HAAR DWT (Level 2) Haar Wavelet Domain Details 2 Higher Resolution Higher Frequency Sign Matching Trend Matching Details 1 Approx 2 D 2 D 1 1 D 1 2 Pattern Found D 1 1 D

13 1. Detection b) Wavelet Comparer: Example CELL (Gattinara Centro, VC), HSDPA Establishment Failure Rate HSDPA EFR Monotonic Step Pattern Input Signal Mismatching Sign Pattern NOT Matching Haar DWT Coefficients Approx 3 D 3 D 2 1 D 2 2 D 1 1,2,3,4 13

14 2. Filtering Challenges Live Input: KPIs 1. Detection 2. Filtering Results RAW data Detected Alarms Filtered Alarms Filtering Objectives Alarm Ranking Aggregation Classification False-Positives Reduction Reduce work load Network Maintenance Efficiency Highlight Alarms of Interest Automate Best Practices Integrate Know-How Network operators are busy, the system is designed to improve efficiency How? 14

15 2. Filtering Intrinsic Detection Ranking Alarm Priority? Deviation from Expected value High Score! Score event i = reference - x reference (e.g. reference = median) CELL (Arluno Casello A4, MI), Dropped Calls i Is this enough for domain experts? No, because: Priority is Affected by Previous Behavior KPI, Cell relevance Not static(change in time).. Lower Score but the Alarm is persistent, It s priority Domain Experts require Smarter Ranking 15

16 2. Filtering Improve Ranking: Example IDEA Tune Score by means of ad-hoc Filters derived by: Domain Experts CELL (Arluno Casello A4, MI), Dropped Calls Persistent Alarm Filter Filtering Increased Score 16

17 2. Filtering IDEA Tune Score by means of ad-hoc Filters derived by: Domain Experts Holiday Alarm Low Network Load Recognize Holidays Dynamic Vacation DB Reduce Ranking CELL (Calcinate,BG) Traffic [Erlangs] Traffic is lower during vacations Lower Score Persistent Alarm CELL (Arluno Casello A4, MI), Dropped Calls Higher Score Evaluate History Time Correlation Increase Ranking Link Alarms Recurring Alarm Isolated Alarm Future Observations Time Correlation Reduce Ranking Reduce False-Positives CELL (Via Napo Torriani, MI), Dropped Calls Single Spike, False-Alarm? Lower Score 17

18 Summary Results 1. Anomalies Detection (green samples) 2. Aggregation, Tagging, Re-Scoring (colored labels) CELL (Paderno-castelletto, MI), Dropped Calls Refined Score = 52 Refined Score = 87 (even if sample is lower) Decreased Score Increased Score False-Positives Less priority Alarm Notified More priority Comparison with State of the Art Commercial tool 18

19 Comparative Assessment (vs. ANTARES) CELL (Paderno-castelletto, MI), Dropped Calls, Analysis by Vodafone ANTARES Vodafone ANTARES State of the Art Commercial tool 45 alarms Many false alarms! -64% False Alarms 19

20 EVoKE Demo Analysis Tool Single Day 28 August2012 North-West Italy Network: 2G, 3G Nokia (31571 cells) 4 KPIs 2G (GSM) 2 KPIs 3G (HSDPA) 4 Detectors 5 Filters Processing Time 16 seconds 20

21 EVoKE Geo Viewer Visual Navigation Shown Example Daily Results 28 August2012 North-West Italy Network: 3G Nokia (18087 cells) KPIs: 3G HSDPA Traffic [Kb] Establishment Failure Rate [%] Spatial Correlation Many alarms, N N Many alarms, Few nearby alarms, Low resolution Higher resolution N Highest Resolution 21

22 Conclusions Resume Scenario: Mobile networks, KPIs Detection: Statistical Methods, Haar Wavelet Filtering: Ranking and Classification Results: EVoKE Future Work Alarms Correlation(Space, Causality) Classification (SVM, Bayesian networks) 22

23 Thank you 23