TheHive 4.1.0 Deployment and Integration with MISP

TheHive 4.1.0 Deployment and Integration with MISP

  1. Deploying (and using) TheHive4 [Part 1]
  2. Building TheHive4 (4.0.5) and configuring MISP, Cortex and Webhooks.
  3. Building the Assemblyline Analyzer for TheHive’s Cortex.
  4. TheHive 4.1.0 Deployment and Integration with MISP

Every few months, StrangeBee puts out an update to TheHive (Security Incident Response Platform). This month they have added Elasticsearch as an index engine to alleviate issues with using Cassandra, and they have integrated support for MISP galaxies as well!

Now Incident Responders using TheHive can export IOCs and Galaxy assignment directly from TheHive to MISP.

Installation of TheHive 4.1.0 on Ubuntu 18.04

A speed-run of installing TheHive from scratch in 12 minutes

System Update and Preparation of Repositories

Let’s first get the updates and the repositories out of the way for the rest of the installation process.

sudo apt update -y && sudo apt upgrade -y
curl -fsSL https://www.apache.org/dist/cassandra/KEYS | sudo apt-key add -
echo "deb http://www.apache.org/dist/cassandra/debian 311x main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
curl https://raw.githubusercontent.com/TheHive-Project/TheHive/master/PGP-PUBLIC-KEY | sudo apt-key add -
echo 'deb https://deb.thehive-project.org release main' | sudo tee -a /etc/apt/sources.list.d/thehive-project.list
sudo apt update -y && sudo apt upgrade -y

Assuming you have already done an update and upgrade, your output versus mine below will likely be the same. If there are extras to update, this should not be an issue.

Updates report at the end of the above code snippet. All repositories have been loaded.

Install Java Runtime Environment

JRE 8 is required for this installation. There are issues with 11 & 12 with Cassandra. So please ensure to install JRE 8 for this build.

sudo apt-get install -y openjdk-8-jre-headless -y
echo JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64" | sudo tee -a /etc/environment
export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64"

Install Cassandra

sudo apt install cassandra -y
cqlsh localhost 9042

You will now need to console into Cassandra to perform a configuration change. Perform the following interactively within CQLSH.

UPDATE system.local SET cluster_name = 'thp' where key='local';
exit;

Then perform a flush using the below from the console…

nodetool flush

And then edit /etc/cassandra/cassandra.yaml to reflect the following requirements:

cluster_name: 'thp'
listen_address: '127.0.0.1' # address for nodes
rpc_address: '127.0.0.1' # address for clients
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          # Ex: "<ip1>,<ip2>,<ip3>"
          - seeds: '127.0.0.1' # self for the first node
data_file_directories:
  - '/var/lib/cassandra/data'
commitlog_directory: '/var/lib/cassandra/commitlog'
saved_caches_directory: '/var/lib/cassandra/saved_caches'
hints_directory: 
  - '/var/lib/cassandra/hints'

Save the changes in your favourite editor, and then restart Cassandra.

sudo systemctl restart cassandra
sudo systemctl enable cassandra

Cassandra should then advertise on port 7000 shortly after the restart.

Port 7000 should come up near instantaneously

Install Elasticsearch

sudo apt install elasticsearch -y
sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch

No further configuration should be required, unless you really wanted to allow your Elasticsearch instance to be accessible externally (not advised or suggested from my perspective).

After a period of about 30 seconds, Elasticsearch should advertise port 9200 on the localhost. You can verify that has occurred through the following command:

netstat -an | grep 9200
Elasticsearch may take up to a minute to become available

Install TheHive

sudo apt-get install thehive4 -y
sudo mkdir /opt/thp/thehive/index
sudo mkdir /opt/thp/thehive/files
sudo chown thehive:thehive -R /opt/thp/thehive/index
sudo chown thehive:thehive -R /opt/thp/thehive/files

Configure TheHive and Restart

Now we can configure TheHive by editing the configuration file appropriately. This will be a basic setup at first, advanced integrations come later under the Integration headings.

In our editor, update the following variables in /etc/thehive/application.conf

## Database configuration
storage {
  ## Cassandra configuration
  # More information at https://docs.janusgraph.org/basics/configuration-refe$
  backend: cql
  hostname: ["127.0.0.1"]
  # Cassandra authentication (if configured)
  // username: "thehive"
  // password: "password"
  cql {
    cluster-name: thp
    keyspace: thehive
  }
  index.search {
    backend: elasticsearch
    hostname: ["127.0.0.1"]
    index-name: thehive
  }
}
## Attachment storage configuration
storage {
  ## Local filesystem
  provider: localfs
  localfs.location: /opt/thp/thehive/files
  ## Hadoop filesystem (HDFS)
  // provider: hdfs
  // hdfs {
  //   root: "hdfs://localhost:10000" # namenode server hostname
  //   location: "/thehive"           # location inside HDFS
  //   username: thehive              # file owner
  // }
}

Save the configuration changes, and then start TheHive from the console

sudo systemctl start thehive
sudo systemctl enable thehive

After about a minute you can issue a grep for port 9000 to show that TheHive is listening and waiting connections. The start-up may take a little bit of time because of the first start, and there is now an Elasticsearch index to be configured as well.

I also very strongly suggest marking both Elasticsearch and TheHive4 from being upgraded as part of apt upgrade. This is next measure is intended to prevent you from bricking your installation through a well intended upgrade which may implement a breaking change.

sudo apt-mark hold elasticsearch
sudo apt-mark hold thehive4

First login should be the same as before, browse to your instance on port 9000, and should be able to login with the below credentials.

There are no changes here for the login screen. Default initial [email protected] account is still present.

First logins to TheHive are still made using the default username and password of [email protected] / secret

TheHive 4.0.1 with Elasticsearch is now operational.

Integration of TheHive with MISP

Integrating MISP with TheHive is relatively straightforward… for this you will need an operational MISP instance which is accessible by TheHive. The following information will need to be updated in /etc/thehive/application.conf

play.modules.enabled += org.thp.thehive.connector.misp.MispModule
misp {
  interval: 1 hour
  servers: [
    {
      name = "TestMISPInstance"            # MISP name
      url = "http://mymispinstance/" # URL or MISP
      auth {
        type = key
        key = "PASTEKEYHERE"             # MISP API key
      }
      wsConfig {}               # HTTP client configuration (SSL and proxy)
    }
  ]
}

Save the configuration file and then restart thehive

sudo systemctl restart thehive

And then check back in TheHive’s interface for the MISP module status icon in the bottom right of your interface.

MISP is configured. NOTE: My MISP instance is behind Cloudflare and as such more advanced configuration is required in the Go HTTP configuration.

Now that MISP is installed, we can install the MISP taxonomies from the Admin menu in the top bar.
Import the taxonomies from the Import Taxonomies button…

The taxonomies will take about a minute to populate. Once done a new screen will be presented with all the tags.

The import may take a little while, but once complete you should see a view of the imported taxonomies in TheHive which correspond with that of MISP. Take care though, sometimes MISP will update these taxonomies, and that may result in an out of sync taxonomy state between MISP and TheHive.

Exporting TheHive Case to MISP

For the purposes of this test, I am using a URL from PhishTank to provide from some intelligence to push to MISP. I have added the URL and manually included external intelligence and some other non-IOC flagged information.

Extracted IOCs and Observables from a single Phishtank report

From here the IOC marked items are pushed to a MISP event through the Export button. With those observables now being replicated into MISP as the following:

Imported IOCs and Observables now in MISP for further processing and distribution

We will look into the next component of assigning Att&ck patterns in TheHive later, however these can also be assigned in MISP too… briefly though you expand the ATT&CK matrix panel and select the appropriate techniques observed per the case.

The case will then be tagged into the galaxies appropriate to those Attack Patterns.

Conclusion

So far, TheHive 4.1.0 has performed pretty well. The true test will come over the following days where I smash alerts and cases through the platform via N8N.

This should give the new Elasticsearch index engine a proper stress test, and be able to test out the webhook authentication which was also added as part of 4.1.0

Happy Incident Responding!

16 Comments

Herman Posted on4:59 pm - March 22, 2021

Thank you for sharing.
I already setup Thehive 4.1. Your posts are really helpful.
I’m eager to see your posts regarding integration with N8N.
Keep up the good work.

Building TheHive4 (4.0.5) and configuring MISP, Cortex and Webhooks. – McHugh Security Posted on10:29 am - March 24, 2021

[…] requirements and significant improvements in index management. Please refer to the updated post here, for up to date instructions for installing TheHive […]

Rikyz Posted on3:22 pm - May 15, 2021

in /etc/thehive/application.conf you have to uncomment line 5 to make everytrhing work and let the hive create the backend on cassandra.
At least on my installation 🙂

    admin Posted on6:08 am - May 15, 2021

    Good pickup, i will update the post accordingly. Thanks.

Stephen Posted on7:00 am - May 27, 2021

I can’t get this to work. I get down to the hive install and edit application.conf; go to start thehive and grep port 9000 and it’s not listening. I’ve gone through this tutorial line by line

    admin Posted on9:41 pm - May 26, 2021

    Do you get an error line in the /var/log/thehive/application.log file?

    I have also found that this can occur when the Java component was not installed.

      Stephen Posted on5:43 am - June 1, 2021

      It worked the second try. Not sure why it didn’t originally, maybe because the java component didn’t install when I thought it did

Bing Posted on1:10 pm - June 14, 2021

Same problem. Nothing in the log file. Cassandra and Elasticsearch come up fine. The Hive refuses to start. Which component of java is potentially missing?

    McHughSecurity Posted on3:50 am - June 14, 2021

    Interesting. I have always used JRE 8 for the installations and that has always worked. I do recall using JRE 11 once before and it not having worked.

    What error do you get from journalctl -xe, or what error statements do you get from the /var/log/thehive/application.log file.

Bing Posted on1:05 pm - June 18, 2021

I’ve solved it. The deb was looking for a data directory in /opt/thp/thehive/ that wasn’t created. Thanks for pointing me to the correct location to figure out the error.

    McHughSecurity Posted on10:10 am - June 18, 2021

    Good to hear mate! I find reviewing the instructions really useful when it doesn’t work to plan. Sometimes there is a minor change which has a larger impact

Stephen Posted on1:59 pm - June 30, 2021

Regarding the “Integration of TheHive with MISP” I’ve edited the config file as mentioned but TheHive doesn’t recognize the MISP instance. Could you elaborate on the “MISP API key” portion. I’ve created a separate user in MISP for this with the correct roles. I am curious of how you are obtaining/creating the API key in MISP for this to work.

    McHughSecurity Posted on2:27 pm - June 30, 2021

    I am writing another piece on MISP integration this week. There is an assumption in my guide that MISP is already working, however TheHive has issues with the way MISP by default handles certificates for https requests.

    If you are getting status errors in the about pages in TheHive then the issue is related to the full chain entry in MISPs http server configuration.

      JulianK Posted on3:22 am - July 21, 2021

      Hello,

      I have tried to fix the status, but I cant. You have any guide more specific for this integration. I create a user with specific role and copy the auth key. The certificate is self sign. But i dont found the error

shraddha Posted on4:17 pm - November 12, 2021

I had 230 events in the misp, when i integrated misp with thehive, just getting 30 alerts,
could you help me with this? i want all 230 alerts.

    Julian Posted on2:29 pm - November 24, 2021

    how do oy integrate misp and the hive? do you have https (tls)?

Leave a Reply

eight − 8 =