Global Analysis of Weak Signals for Enterprise Event Detection (GAWSEED)

Project Description

The Global Analysis of Weak Signals for Enterprise Event Detection (GAWSEED) project studed weak signals across multiple large-enterprise datasets looking for signs of malicious activity so small they may be passed over by a single enterprise’s operational staff. GAWSEED had three goals to meet this challenge:

  • We created new security-event sensors by analyzing structures in data sources using insights about protocols and host-based information sources. Analysis of these structures developed new methods to expose security events even in weak signals, and evaluate use of corroborative information to further amplify them.

  • These signals and supporting information will serve as features for machine-learning-augmented algorithms to find, classify, and prioritize discovered security events.

  • To draw upon sensor data distributed across multiple enterprises, we developed inter-enterprise sharing and control protocols. These protocols addressed challenges in distributed computation and constrained communication in the face of soemtimes conflicting policies about privacy and sharing.

GAWSEED is part of ANT Lab at USC/ISI (PIs: Wes Hardaker and John Heidemann in the networking division, and Aram Galystan from the AI division. It is joint work with researchers at Parsons Corporation. It is supported by DARPA as part of the CHASE program.

Released Results

The DARPA/CHASE project has produced a public facing GAWSEED Internet Feed of Threats (GiFT) webpage that shows daily updated results from the GAWSEED project. Available on the GiFT site are daily downloadable Indicators of Compromise (IoCs) and other continual analysis results. Some of the content is restricted to account holders, so reach out to me if you’re interested in an account that provides you access to the browsable analytic sections.

People

The following people contributed to the GAWSEED project at some point over time:

  • Abdulla Alwabel, PhD student (USC CS Dept. and ISI)
  • Calvin Ardi, USC CS PhD graduate (2020) (USC CS Dept. and ISI)
  • Michael Baer, researcher (PARSONS)
  • Genevieve Bartlett, researcher (USC CS Dept. and ISI)
  • Valentino Crespi, research staff (USC/ISI)
  • Aram Galstyan, research director and research associate professor (USC/ISI)
  • Wes Hardaker, PI on this project, researcher (USC/ISI)
  • Hrayr Harutyunyan, PhD student (USC/ISI)
  • John Heidemann, co-PI on this project, project leader and professor (USC/ISI)
  • Haoyu Jiang, graduate research assistant (USC Viterbi)
  • Suresh Krishnaswamy, researcher (PARSONS)
  • Wayne Morrison, researcher (PARSONS)
  • Russ Mundy, researcher (PARSONS)
  • Sandy Murphy, researcher (PARSONS)
  • Yuri Pradkin, researcher (USC/ISI)
  • Spencer Stingley, USC MS graduate (2023), now member of technical staff (USC/ISI)
  • Robert Story, Computer Analyst (USC/ISI)
  • Erin Szeto, Masters student (USC/ISI)

Publications

  • Abdul Qadeer and John Heidemann 2021. Efficient Processing of Streaming Data using Multiple Abstractions. Proceedings of the IEEE International Conference on Cloud Computing (Virtual, Sep. 2021), 157–167. [DOI] [PDF] Details
  • Abdul Qadeer and John Heidemann 2020. Plumb: Efficient Stream Processing of Multi-User Pipelines. Software—Practice and Experience. 51, 2 (2020), 385–408. [DOI] [PDF] Details
  • Wes Hardaker 2020. GAWSEED. ISI Research Day. Details
  • Hrayr Harutyunyan, Daniel Moyer, Hrant Khachatrian, Greg Ver Steeg and Aram Galstyan 2019. Efficient Covariance Estimation from Temporal Data. arXiv preprint arXiv:1905.13276. (2019). Details
  • Greg Ver Steeg, Hrayr Harutyunyan, Daniel Moyer and Aram Galstyan 2019. Fast structure learning with modular regularization. Advances in Neural Information Processing Systems 32. H. Wallach, H. Larochelle, A. Beygelzimer, F. d’ Alché-Buc, E. Fox, and R. Garnett, editors. Curran Associates, Inc. 15567–15577. [PDF] Details

For related publications, please see the ANT publications web page.

Software

pip3 installable software:

  • pyfsdb
  • gawseed-processing
  • gawseed-threat-feed-tools

  • gawseed-tcorex

See also the ANT software web page.

Acknowledgments

This research is based upon work supported in part by DARPA, via W911NF-16-1-0575, and the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via 2016-16041100004. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DARPA, ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.