Session:Building a stream-based NLP (Natural Language Processing) app to monitor vulnerabilities realtime

From MCH2022 wiki
Jump to navigation Jump to search
Description We'll set up a stream-based Python app to monitor new vulnerabilities by using NLP (Natural Language Processing) in realtime
Website(s)
Type
Kids session No
Keyword(s) software
Tags faust, nlp, kafka, elasticsearch, python, vulnerabilities
Processing village Village:Area42
Person organizing User:D3vzer0
Language en - English
en - English
Other sessions... ... further results
Subtitle Hands-On
Starts at 2022-07-24T14:00:00.000Z
Ends at 2022/07/24 15:00
Duration 60 minutes
Location Village:Area42

We'll be experimenting with some basic NLP to monitor when vulnerabilities may go trending. Using Faust (stream processing) we'll monitor RSS feeds, tweets and the NVD database and extract important keywords using basic Natural Language Processing. The results will be saved in ElasticSearch where you'll be able to create fancy graphs about what's currently trending!

Virtual machines running most of the components will be made available. The VMs will run: - Kafka - ElasticSearch - Kibana - Postgres - Miniflux

All the materials: - github.com/d3vzer0/mch2022-workshop-nlp (the NLP api) - github.com/d3vzer0/mch2022-workshop-streaming (the presentation and Faust engine) - github.com/d3vzer0/mch2022-workshop-cloud (the ansible/terraform deployment)

PS. Will be to facilitate about 6 people. I'll be able to organise multiple sessions if there is more interest :)