Skip to content

McHughSecurity

  • Home
  • Blog
  • Security Operations
  • DFIR
    • Reverse Engineering
      • Dynamic Analysis
      • Static Analysis
  • Vuln. Mgmt
  • Intelligence
    • Open-Source Intelligence
    • Threat Intelligence
      • AIL Framework
      • MISP – Open Source Threat Intelligence Platform
      • OpenCTI
  • Threat Modelling
    • Attack Libraries
    • Attack Trees
    • Mitigating Techniques
    • SaaS Threat Modelling
    • Threat Models
    • Tools
    • Validating Threat Mitigations
  • Toggle search form
  • OSINT for Threat Intelligence Blog
  • Deploying (and using) TheHive4 [Part 1] Build
  • Hardening TheHive4 and Cortex for public deployment Build
  • Extracting RAM from VirtualBox session Digital Forensics & Incident Response
  • Building a Cuckoo Sandbox Build
  • Building a MISP Threat Feed Aggregator Blog
  • Auto-updating Ubuntu 20.04 in less than 2 minutes Operate
  • TraceLabs Missing Persons 11th of April 2020 Open-Source Intelligence

TheHive 4.1.0 Deployment and Integration with MISP

Posted on March 20, 2021 By A.McHugh 16 Comments on TheHive 4.1.0 Deployment and Integration with MISP
  1. Deploying (and using) TheHive4 [Part 1]
  2. Building TheHive4 (4.0.5) and configuring MISP, Cortex and Webhooks.
  3. Building the Assemblyline Analyzer for TheHive’s Cortex.
  4. TheHive 4.1.0 Deployment and Integration with MISP

Every few months, StrangeBee puts out an update to TheHive (Security Incident Response Platform). This month they have added Elasticsearch as an index engine to alleviate issues with using Cassandra, and they have integrated support for MISP galaxies as well!

Now Incident Responders using TheHive can export IOCs and Galaxy assignment directly from TheHive to MISP.

Installation of TheHive 4.1.0 on Ubuntu 18.04

A speed-run of installing TheHive from scratch in 12 minutes

System Update and Preparation of Repositories

Let’s first get the updates and the repositories out of the way for the rest of the installation process.

sudo apt update -y && sudo apt upgrade -y
curl -fsSL https://www.apache.org/dist/cassandra/KEYS | sudo apt-key add -
echo "deb http://www.apache.org/dist/cassandra/debian 311x main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
curl https://raw.githubusercontent.com/TheHive-Project/TheHive/master/PGP-PUBLIC-KEY | sudo apt-key add -
echo 'deb https://deb.thehive-project.org release main' | sudo tee -a /etc/apt/sources.list.d/thehive-project.list
sudo apt update -y && sudo apt upgrade -y

Assuming you have already done an update and upgrade, your output versus mine below will likely be the same. If there are extras to update, this should not be an issue.

Updates report at the end of the above code snippet. All repositories have been loaded.

Install Java Runtime Environment

JRE 8 is required for this installation. There are issues with 11 & 12 with Cassandra. So please ensure to install JRE 8 for this build.

sudo apt-get install -y openjdk-8-jre-headless -y
echo JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64" | sudo tee -a /etc/environment
export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64"

Install Cassandra

sudo apt install cassandra -y
cqlsh localhost 9042

You will now need to console into Cassandra to perform a configuration change. Perform the following interactively within CQLSH.

UPDATE system.local SET cluster_name = 'thp' where key='local';
exit;

Then perform a flush using the below from the console…

nodetool flush

And then edit /etc/cassandra/cassandra.yaml to reflect the following requirements:

cluster_name: 'thp'
listen_address: '127.0.0.1' # address for nodes
rpc_address: '127.0.0.1' # address for clients
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          # Ex: "<ip1>,<ip2>,<ip3>"
          - seeds: '127.0.0.1' # self for the first node
data_file_directories:
  - '/var/lib/cassandra/data'
commitlog_directory: '/var/lib/cassandra/commitlog'
saved_caches_directory: '/var/lib/cassandra/saved_caches'
hints_directory: 
  - '/var/lib/cassandra/hints'

Save the changes in your favourite editor, and then restart Cassandra.

sudo systemctl restart cassandra
sudo systemctl enable cassandra

Cassandra should then advertise on port 7000 shortly after the restart.

Port 7000 should come up near instantaneously

Install Elasticsearch

sudo apt install elasticsearch -y
sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch

No further configuration should be required, unless you really wanted to allow your Elasticsearch instance to be accessible externally (not advised or suggested from my perspective).

After a period of about 30 seconds, Elasticsearch should advertise port 9200 on the localhost. You can verify that has occurred through the following command:

netstat -an | grep 9200
Elasticsearch may take up to a minute to become available

Install TheHive

sudo apt-get install thehive4 -y
sudo mkdir /opt/thp/thehive/index
sudo mkdir /opt/thp/thehive/files
sudo chown thehive:thehive -R /opt/thp/thehive/index
sudo chown thehive:thehive -R /opt/thp/thehive/files

Configure TheHive and Restart

Now we can configure TheHive by editing the configuration file appropriately. This will be a basic setup at first, advanced integrations come later under the Integration headings.

In our editor, update the following variables in /etc/thehive/application.conf

## Database configuration
storage {
  ## Cassandra configuration
  # More information at https://docs.janusgraph.org/basics/configuration-refe$
  backend: cql
  hostname: ["127.0.0.1"]
  # Cassandra authentication (if configured)
  // username: "thehive"
  // password: "password"
  cql {
    cluster-name: thp
    keyspace: thehive
  }
  index.search {
    backend: elasticsearch
    hostname: ["127.0.0.1"]
    index-name: thehive
  }
}
## Attachment storage configuration
storage {
  ## Local filesystem
  provider: localfs
  localfs.location: /opt/thp/thehive/files
  ## Hadoop filesystem (HDFS)
  // provider: hdfs
  // hdfs {
  //   root: "hdfs://localhost:10000" # namenode server hostname
  //   location: "/thehive"           # location inside HDFS
  //   username: thehive              # file owner
  // }
}

Save the configuration changes, and then start TheHive from the console

sudo systemctl start thehive
sudo systemctl enable thehive

After about a minute you can issue a grep for port 9000 to show that TheHive is listening and waiting connections. The start-up may take a little bit of time because of the first start, and there is now an Elasticsearch index to be configured as well.

I also very strongly suggest marking both Elasticsearch and TheHive4 from being upgraded as part of apt upgrade. This is next measure is intended to prevent you from bricking your installation through a well intended upgrade which may implement a breaking change.

sudo apt-mark hold elasticsearch
sudo apt-mark hold thehive4

First login should be the same as before, browse to your instance on port 9000, and should be able to login with the below credentials.

There are no changes here for the login screen. Default initial [email protected] account is still present.

First logins to TheHive are still made using the default username and password of [email protected] / secret

TheHive 4.0.1 with Elasticsearch is now operational.

Integration of TheHive with MISP

Integrating MISP with TheHive is relatively straightforward… for this you will need an operational MISP instance which is accessible by TheHive. The following information will need to be updated in /etc/thehive/application.conf

play.modules.enabled += org.thp.thehive.connector.misp.MispModule
misp {
  interval: 1 hour
  servers: [
    {
      name = "TestMISPInstance"            # MISP name
      url = "http://mymispinstance/" # URL or MISP
      auth {
        type = key
        key = "PASTEKEYHERE"             # MISP API key
      }
      wsConfig {}               # HTTP client configuration (SSL and proxy)
    }
  ]
}

Save the configuration file and then restart thehive

sudo systemctl restart thehive

And then check back in TheHive’s interface for the MISP module status icon in the bottom right of your interface.

MISP is configured. NOTE: My MISP instance is behind Cloudflare and as such more advanced configuration is required in the Go HTTP configuration.

Now that MISP is installed, we can install the MISP taxonomies from the Admin menu in the top bar.
Import the taxonomies from the Import Taxonomies button…

The taxonomies will take about a minute to populate. Once done a new screen will be presented with all the tags.

The import may take a little while, but once complete you should see a view of the imported taxonomies in TheHive which correspond with that of MISP. Take care though, sometimes MISP will update these taxonomies, and that may result in an out of sync taxonomy state between MISP and TheHive.

Exporting TheHive Case to MISP

For the purposes of this test, I am using a URL from PhishTank to provide from some intelligence to push to MISP. I have added the URL and manually included external intelligence and some other non-IOC flagged information.

Extracted IOCs and Observables from a single Phishtank report

From here the IOC marked items are pushed to a MISP event through the Export button. With those observables now being replicated into MISP as the following:

Imported IOCs and Observables now in MISP for further processing and distribution

We will look into the next component of assigning Att&ck patterns in TheHive later, however these can also be assigned in MISP too… briefly though you expand the ATT&CK matrix panel and select the appropriate techniques observed per the case.

The case will then be tagged into the galaxies appropriate to those Attack Patterns.

Conclusion

So far, TheHive 4.1.0 has performed pretty well. The true test will come over the following days where I smash alerts and cases through the platform via N8N.

This should give the new Elasticsearch index engine a proper stress test, and be able to test out the webhook authentication which was also added as part of 4.1.0

Happy Incident Responding!

Related

Build, Digital Forensics & Incident Response Tags:#misp, cortex, elasticsearch, thehive

Post navigation

Previous Post: Building the Assemblyline Analyzer for TheHive’s Cortex.
Next Post: Using MISP in an air-gapped environment

Related Posts

  • Building a Cuckoo Malware Analysis Server Digital Forensics & Incident Response
  • [Part 1] Building a Threat Integration and Testing Lab Build
  • Building a parallel-analysis Cuckoo server Digital Forensics & Incident Response
  • Deploying (and using) TheHive4 [Part 1] Build
  • [Part 2] Building a Threat Integration and Testing Lab – Elastic Cloud Enterprise (On-Premises) Build
  • Hardening TheHive4 and Cortex for public deployment Build

Comments (16) on “TheHive 4.1.0 Deployment and Integration with MISP”

  1. Herman says:
    March 22, 2021 at 4:59 pm

    Thank you for sharing.
    I already setup Thehive 4.1. Your posts are really helpful.
    I’m eager to see your posts regarding integration with N8N.
    Keep up the good work.

    Reply
  2. Pingback: Building TheHive4 (4.0.5) and configuring MISP, Cortex and Webhooks. - McHugh Security
  3. Rikyz says:
    May 15, 2021 at 3:22 pm

    in /etc/thehive/application.conf you have to uncomment line 5 to make everytrhing work and let the hive create the backend on cassandra.
    At least on my installation 🙂

    Reply
    1. admin says:
      May 15, 2021 at 6:08 am

      Good pickup, i will update the post accordingly. Thanks.

      Reply
  4. Stephen says:
    May 27, 2021 at 7:00 am

    I can’t get this to work. I get down to the hive install and edit application.conf; go to start thehive and grep port 9000 and it’s not listening. I’ve gone through this tutorial line by line

    Reply
    1. admin says:
      May 26, 2021 at 9:41 pm

      Do you get an error line in the /var/log/thehive/application.log file?

      I have also found that this can occur when the Java component was not installed.

      Reply
      1. Stephen says:
        June 1, 2021 at 5:43 am

        It worked the second try. Not sure why it didn’t originally, maybe because the java component didn’t install when I thought it did

        Reply
  5. Bing says:
    June 14, 2021 at 1:10 pm

    Same problem. Nothing in the log file. Cassandra and Elasticsearch come up fine. The Hive refuses to start. Which component of java is potentially missing?

    Reply
    1. McHughSecurity says:
      June 14, 2021 at 3:50 am

      Interesting. I have always used JRE 8 for the installations and that has always worked. I do recall using JRE 11 once before and it not having worked.

      What error do you get from journalctl -xe, or what error statements do you get from the /var/log/thehive/application.log file.

      Reply
  6. Bing says:
    June 18, 2021 at 1:05 pm

    I’ve solved it. The deb was looking for a data directory in /opt/thp/thehive/ that wasn’t created. Thanks for pointing me to the correct location to figure out the error.

    Reply
    1. McHughSecurity says:
      June 18, 2021 at 10:10 am

      Good to hear mate! I find reviewing the instructions really useful when it doesn’t work to plan. Sometimes there is a minor change which has a larger impact

      Reply
  7. Stephen says:
    June 30, 2021 at 1:59 pm

    Regarding the “Integration of TheHive with MISP” I’ve edited the config file as mentioned but TheHive doesn’t recognize the MISP instance. Could you elaborate on the “MISP API key” portion. I’ve created a separate user in MISP for this with the correct roles. I am curious of how you are obtaining/creating the API key in MISP for this to work.

    Reply
    1. McHughSecurity says:
      June 30, 2021 at 2:27 pm

      I am writing another piece on MISP integration this week. There is an assumption in my guide that MISP is already working, however TheHive has issues with the way MISP by default handles certificates for https requests.

      If you are getting status errors in the about pages in TheHive then the issue is related to the full chain entry in MISPs http server configuration.

      Reply
      1. JulianK says:
        July 21, 2021 at 3:22 am

        Hello,

        I have tried to fix the status, but I cant. You have any guide more specific for this integration. I create a user with specific role and copy the auth key. The certificate is self sign. But i dont found the error

        Reply
  8. shraddha says:
    November 12, 2021 at 4:17 pm

    I had 230 events in the misp, when i integrated misp with thehive, just getting 30 alerts,
    could you help me with this? i want all 230 alerts.

    Reply
    1. Julian says:
      November 24, 2021 at 2:29 pm

      how do oy integrate misp and the hive? do you have https (tls)?

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

five × 2 =

Recent Posts

  • Auto-updating Ubuntu 20.04 in less than 2 minutes
  • Feeding Analysis Information Leak (AIL) Framework
  • An Introduction to Threat Intelligence
  • Deploying MISP on DigitalOcean or Vultr Cloud Hosting
  • Building CCCS’ AssemblyLine for Static Analysis

Search

Recent Posts

  • Auto-updating Ubuntu 20.04 in less than 2 minutes
  • Feeding Analysis Information Leak (AIL) Framework
  • An Introduction to Threat Intelligence
  • Deploying MISP on DigitalOcean or Vultr Cloud Hosting
  • Building CCCS’ AssemblyLine for Static Analysis

Archives

  • April 2022
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • September 2020
  • April 2020
  • March 2020
  • January 2020
  • May 2019
  • Building TheHive4 (4.0.5) and configuring MISP, Cortex and Webhooks. Digital Forensics & Incident Response
  • Security Orchestration with Shuffle.io Design
  • Auto-updating Ubuntu 20.04 in less than 2 minutes Operate
  • TraceLabs Missing Persons 11th of April 2020 Open-Source Intelligence
  • Implementing Elastic Cloud and using Elastic Security Build
  • Hardening TheHive4 and Cortex for public deployment Build
  • Building a Cuckoo Sandbox Build
  • [Part 2] Building a Threat Integration and Testing Lab – Elastic Cloud Enterprise (On-Premises) Build

Categories

  • AIL Framework
  • Blog
  • Build
  • Design
  • Digital Forensics & Incident Response
  • Frameworks
  • Intelligence
  • MISP – Open Source Threat Intelligence Platform
  • Open-Source Intelligence
  • Operate
  • Security Operations
  • Static Analysis
  • Threat Intelligence

Copyright © 2022 McHughSecurity.

Powered by PressBook News Dark theme