Eastern Oak

Using gdb to debug a stack overflow in my Rust test

I'm working on a Rust MQTT library called tjiftjaf. The other day I refactored some code and tests started failing with a stack overflow.

$ cargo test
...
thread 'test::test_mqtt_binding_decoding_packets' (127197) has overflowed its stack
fatal runtime error: stack overflow, aborting
error: test failed, to rerun pass `--lib`

Caused by:
  process didn't exit successfully: `/home/auke/projects/tjiftjaf/target/debug/deps/tjiftjaf-23491dbdc1ea146f` (signal: 6, SIGABRT: process abort signal)

The error doesn't not include a backtrace, making it hard to understand what caused the stack overflow.

Sprinkling the code with println! didn't help, nor configuring RUST_BACKTRACE to produce a more helpful backtrace.

Then, I realized that cargo test builds the test suite into a binary and I can use gdb on that binary.

The path to the binary is included in the error message. It is /home/auke/projects/tjiftjaf/target/debug/deps/tjiftjaf-23491dbdc1ea146f.

So I fired up gdb:

$ gdb /home/auke/projects/tjiftjaf/target/debug/deps/tjiftjaf-23491dbdc1ea146f
(gdb) run
...
[Thread 0x7ffff67f56c0 (LWP 127997) exited]

Thread 27 "test::test_mqtt" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff69f66c0 (LWP 127996)]
0x00005555555cd137 in core::convert::{impl#3}::into<tjiftjaf::packet::connack::ConnAck, alloc::vec::Vec<u8, alloc::alloc::Global>> (self=...)
    at /home/auke/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/convert/mod.rs:777
777	    fn into(self) -> U {

The stack overflow originates from the core::convert::Into::into().

What called into (pun intended) here?

The command up shows the previous frame in the call stack which may show more details. /u/eras on Reddit suggested backtrace as alternative.

(gdb) up
#1  0x00005555555dda0f in tjiftjaf::packet::connack::{impl#2}::from (value=...) at src/packet/connack.rs:44
44	        value.into()

Expand the surrounding code with list to see the context:

(gdb) list
39	    }
40	}
41	
42	impl From<ConnAck> for Vec<u8> {
43	    fn from(value: ConnAck) -> Self {
44	        value.into()
45	    }
46	}
47	
48	impl From<ConnAck> for Packet {

So Vec<u8>::from() calls ConnAck.into(). And that calls Vec<u8>::from() again, as previous frame on the stack shows. In other words: recursion leads to a stack overflow.

(gdb) up
#2  0x00005555555cd158 in core::convert::{impl#3}::into<tjiftjaf::packet::connack::ConnAck, alloc::vec::Vec<u8, alloc::alloc::Global>> (self=...)
    at /home/auke/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/convert/mod.rs:778
778	        U::from(self)
(gdb) list
773	    /// That is, this conversion is whatever the implementation of
774	    /// <code>[From]&lt;T&gt; for U</code> chooses to do.
775	    #[inline]
776	    #[track_caller]
777	    fn into(self) -> U {
778	        U::from(self)
779	    }
780	}
781	
782	// From (and thus Into) is reflexive

The fix is easy: I need to convert ConnAck's inner type to Vec<u8>.

diff --git a/src/packet/connack.rs b/src/packet/connack.rs
index c2aa326..596ae21 100644
--- a/src/packet/connack.rs
+++ b/src/packet/connack.rs
@@ -41,7 +41,7 @@ impl Frame for ConnAck {

 impl From<ConnAck> for Vec<u8> {
     fn from(value: ConnAck) -> Self {
-        value.into()
+        value.inner.to_vec()
     }
 }
`