Building a new foundation for security

Bitbison was born from personal pain with the runtime security ecosystem and everything that touches it, including supply chain and non-deterministic agentic workloads that are effectively insider threats. When I started this journey, 0-day exploits were expensive, agents were on the fringe and major supply chain attacks were uncommon. The current generation of frontier models changed that quickly. 0-days are now a commodity, major supply chain attacks are a regular occurrence and every organization now hosts a potential insider threat in the form of AI agents that lack sufficient security controls.

We can no longer operate production systems without a significant improvement to security observability and policy. With the support of our investors, angels and advisors, we have brought together a world-class team to share a modern re-imagining of security. We are now opening early access for anyone interested in playing an active role in shaping it. This post is the story of why we started and what we chose to build first.

The origin story

The day we got popped

Log4Shell was disclosed in December 2021 and impacted hundreds of millions of systems. One of the organizations I was involved with was among them. A build server had a VPN misconfiguration and was accidentally exposed to the internet. We were running current runtime security products alongside file integrity monitoring and malware scanners. There was no detection. A CPU utilization alert we decided not to ignore is what brought the breach to our attention. We were not expecting a breach at all.

We logged into the system and identified a Bitcoin miner. There was nothing else to go on. Aside from the miner, no new files were present. There was no indication of further persistence and no log of what else the attacker did with the instance. Our build and packaging environment was isolated, so we could determine that neither our source nor our external packages were compromised. We could not determine whether anything was exfiltrated, whether the attack was targeted or whether there were attempts at lateral movement. Leaving no evidence behind is easy. I learned first hand during forensics that the default disposition of runtime security products is to log almost nothing. This holds even when there is a detection. We re-imaged the instance and crossed our fingers.

This always bothered me.

The insider threat problem

Another organization I was involved with went through several rounds of lay-offs in the economic aftermath of COVID. I inherited a severely understaffed security organization with insufficient tooling, through no fault of its own. I had assumed that decades of improvements to systems observability for performance had been extended to runtime security monitoring. That was not the case.

Most engineers disabled or circumvented the runtime monitoring products because of their impact on system performance. When enabled, observability was limited. I was surprised to learn the modern approach to runtime security is still built on whitelists. As with the Log4Shell episode, the observability was not there to answer what did or did not happen across thousands of endpoints and servers.

Our small security team was tasked with identifying insider threats. One employee exfiltrated intellectual property. Telemetry was minimal; aside from an anomalous increase in outbound network traffic, there was no way to scope the exfiltration with certainty. The tooling could not observe everything all the time. The performance trade-off meant the best you could hope for was whitelist monitoring, which is ineffective when the monitored systems run arbitrary workloads.

Solving the observability problem

We started by revisiting the observability problem. We could not have detected activity that was never observed. No audit is possible when nothing is logged. Existing audit systems do not meet this bar; full capture slows workloads by 2x to 8x. The problem is logging enough to understand cause and effect, at production scale, with minimal impact to the runtime.

This is a deep problem with close to 20 years of active research behind it, known as full systems provenance. It breaks into two parts:

• Monitoring all system activity, all the time, so the cause-and-effect relationship between events is preserved.
• Storing that information so an audit is always possible.

R. Sekar, Hanke Kimm and Rohit Aich measured how existing systems handle the first problem. Their eAudit paper at the 2024 IEEE Symposium on Security and Privacy benchmarks the established collection systems including sysdig, auditd and CamFlow. Under sustained workloads every one of them drops events. Complete capture is only reached after the workload itself has been slowed down to a crawl.

Figure 2 from eAudit: A Fast, Scalable and Deployable Audit Data Collection System by R. Sekar, Hanke Kimm and Rohit Aich (IEEE Symposium on Security and Privacy, 2024). The fraction of events dropped by each system, plotted against how much the benchmark had to be slowed down before capture became complete.

We built Bitbison to remove this trade-off. Below is a full-fidelity capture of a 16-core parallel compilation of the Ubuntu Linux kernel. The industry standard solution loses 50% of events at 100% CPU cost and 9.2 GB of storage. Bitbison loses 0% of events at 6% CPU and 80 MB.

The same holds on servers. Below is a production workloads with the default coverage of a leading runtime monitoring solution over a four hour period. The most data-rich agent on the market spikes to 129% of a core and records 56,587 events, 7.8% of what happened. Bitbison stays under 8% of a single core and records all 722,683 events.

Even without attempting audit-grade capture, existing systems carry workload-impacting CPU and memory costs while providing near 0% runtime observability.

Detection built on deep observability

Bitbison now runs on dozens of production systems and developer laptops and is actively processing billions of events. It is already deployed in production at a leading semiconductor company. It provides complete causal coverage of all activity on the systems it runs on, at a fraction of the CPU and memory of alternatives. Going back to the origin story, this solves the first major pain point. We are no longer blind. Had Bitbison existed then, one query would have told us exactly what happened during the Log4Shell breach. The insider exfiltration would have been scoped the same way.

What a detection looked like before Bitbison

In one of our red team scenarios an account was compromised. The attacker logged into the system, established a reverse shell with netcat, then downloaded and executed malware. We purposefully chose a sequence we knew would be detectable, to measure the visibility we would be given.

Below is what the existing EDR (endpoint detection and response) solution surfaced to us.

An analyst left to investigate this detection has nothing beyond this graph to go on. Note that the attack sequence had to be modified to be detectable in the first place. Even with a detection, all we know is:

• A legitimate session was started.
• A reverse shell was probably launched through the ncat command listening on port 80.
• The reverse shell launched a shell.
• The shell then executed curl to download a Python script.
• That Python script was probably executed.

The story is incomplete. The graph cannot say what connected to the reverse shell, what files were touched or whether anything left the host. In this scenario Bitbison ingests signals from third party runtime products and augments them with its own causally complete system capture.

What a detection looks like with Bitbison

Below is the same detection in Bitbison.

This signal is confirmed as a true positive: a root bind shell on TCP/80 driven by an external operator, a credential-stealing script fetched from an external host and a beacon out over HTTPS. The report also states what did not happen; no SSH key reads, no /etc/shadow reads and no persistence survived the session. Every conclusion links back to recorded events. The analysis is backed by a formal verification framework built on an in-house model checker to ensure consistent coverage of the investigation.

The scope is both complete and exact.

Evolving policy

The past

The birth of the EDR and runtime security category is attributed to a shift from the signature matching of antivirus products to what is now called "behavioral analysis". In practice, behavioral analysis is pattern matching on process names and system call arguments over an already incomplete stream of data; runtime monitoring's performance bottleneck limits it to coarse-grained events. Some machine learning approaches have made it to production but they:

• Are limited by the fundamental observability problem; you cannot write rules against things never observed.
• Are high latency and unable to detect breaches in real time.

For example, a policy today to alert on an untrusted executable may be dozens of rules that look similar to this:

- rule: Execution from Temp Directory
  desc: >
    Detect a binary being executed from /tmp or /var/tmp,
    a common pattern for dropped payloads and post-exploitation tooling.
  condition: >
    spawned_process
    and (proc.exepath startswith /tmp/
         or proc.exepath startswith /var/tmp/)
  output: >
    Program executed from temp directory
    (user=%user.name command=%proc.cmdline exe=%proc.exepath
    parent=%proc.pname container_id=%container.id image=%container.image.repository)
  priority: WARNING
  tags: [mitre_execution, T1059, filesystem]

In other words, if a process is spawned and its executable begins with /tmp or /var/tmp, trigger an alert. This rule is typically evaded. Even commodity bots and exploits will avoid such directories. In the Log4Shell breach described earlier, detection was evaded because the attacker executed malware from a service directory.

Fundamentally, modern policies are little more than allow and deny lists.

A better approach

Pattern matching has a place but a stronger approach can replace most of its use cases. Bitbison makes full causal logging practical. This means it can reason statefully over the system and its history, including provenance: how things came to be on the system. Once questions like "who created this file?" and "which network endpoints influenced its creation?" can be answered in near real time, policies can be defined against the state of the system and flows of behavior, much closer to their true intention. In the example above, the intention is to only permit trusted programs to execute. Trust is defined by the provenance of the file: who installed it and from where. Bitbison can specify this directly.

alert on $e:fspath →[execute] _ unless (
 pkgmgr →[proc] $i:thd, $i →[write] $e, conns($i) ⊆ repos );

In other words, the policy directly specifies "alert on any executable whose origin and contents cannot be exclusively traced back to the package manager installing from an official repository.

There are no false positives here and there is no list to evade. The origin of everything is always available to query against.

What made this possible

It was not easy. We went back to fundamentals and rethought the observability data model itself: data structures, memory management, allocation and data representation both in memory and on disk. Bitbison pairs several novel approaches to that data model with current techniques in vectorization, concurrent data structures and storage representation. It is the result of the collective experience of our whole careers.

What comes next

Bitbison for Servers and Bitbison for Builds are available today for design partners, all under the umbrella of a holistic security platform that begins by securing your most important assets and the workloads they run on.

You will work closely with our team and have a direct impact on our capabilities and supported use cases. The surface area is broad. As the team scales out, we imagine a single platform covering endpoints, servers, agents, developer workstations and more. The more complete the provenance graph, the stronger the security guarantees that can be made.

Sign up for early access or request a demo.