Ruby and D Interop: Part 2
Whilst I was experimenting with D and RubyFFI I had wondered if there was an easier way… writing out the bindings was becoming quite cumbersome. So I of course turned to the introspective features of D, not unfamiliar to those well versed in Ruby’s metaprogramming trickery.
D does not have particularly strong runtime reflection capabilities but you can determine a lot of what you need at compile time.
In this post I will walk you through my D Ruby FFI Project. This project aims to grease the wheels of writing native code in D for use in Ruby by automagically generating bindings for you on demand as well as providing some other useful things.
If you have not read the previous post in this series you can catch up here.
Using the rubyffi mixin
For the sake of simple examples compilation instructions assume you have copied rubyffi.d from the repository to the same location as the file you’re compiling.
Let’s take my previous example
//fizz.d
import std.stdio : writeln;
import std.conv : to;
export extern(C) void fizzBuzz(int num) {
auto result =
num % 15 == 0 ? "FizzBuzz" :
num % 5 == 0 ? "Buzz" :
num % 3 == 0 ? "Fizz" :
"Boring" ;
//The below line performs a memory allocation.
(num.to!string ~ " is " ~ result).writeln;
}With the rubyffi mixin this becomes
//fizz.d
import rubyffi;
import std.stdio : writeln;
import std.conv : to;
mixin rubyffi!__MODULE__;
export extern(C) {
@Ruby void fizzBuzz(int num) {
auto result =
num % 15 == 0 ? "FizzBuzz" :
num % 5 == 0 ? "Buzz" :
num % 3 == 0 ? "Fizz" :
"Boring" ;
//The below line performs a memory allocation.
(num.to!string ~ " is " ~ result).writeln;
}
}Understandably this seems like more work for not much gain but bear with me! This becomes so much more useful the more functions you have.
Compile it with dmd -shared -defaultlib=libphobos2.so fizz.d -of=libfizz.so then we want to run dmd -o- fizz.d -version=bindgen 2> fizzbuzz.rb to automagically generate the correct bindings for our platform!
If we inspect fizzbuzz.rb we will see that the correct bindings have been generated.
require 'ffi'
module FIZZ
extend FFI::Library
ffi_lib "#{Dir.pwd}/libfizz.so"
attach_function :rt_init, [], :int
attach_function :rt_term, [], :int
attach_function :fizzBuzz, [:int], :void
end
FIZZ::rt_init
at_exit do
FIZZ::rt_term
endWhich we can then import and run.
#!/usr/bin/ruby
require 'ffi'
require './fizzbuzz'
FIZZ.fizzBuzz(10)
FIZZ.fizzBuzz(15)
FIZZ.fizzBuzz(9)Thanks to the rubyffi mixin the D code written is actually platform agnostic. There is no need to add in the Windows DLL hooks from the previous entry in this series since the mixin takes care of that for us.
D mixins work in a similar way to Ruby mixins. The code generation machinery is seamlessly blended with your own code when the bindgen version is specified to the DMD compiler with -version. If bindgen is not specified the code is completely skipped over during the compilation process. Only mixing in the @Ruby user-defined attribute and DllMain in the case of a Windows build.
The @Ruby attribute
The @Ruby attribute provides a convenient way for the rubyffi code to introspect into our code at compile time. We can also use it to specify a ‘friendly’ name for our function so the function can have a different name on the Ruby side to avoid conflicts, this is as simple as writing @Ruby(“newName”).
The introspection code itself is quite simple
immutable fun = "\tattach_function %s:%s, [%s], :%s\n";
static foreach(i, func; getSymbolsByUDA!(mixin(mod), Ruby)) {
static if(functionLinkage!func == "C") {
res ~= fun._fmt(_friend!func, __traits(identifier, func), _params!(Parameters!func), _rt!(ReturnType!func));
}
}First it scans through the code for all @Ruby functions within the module supplied. We only care about the C externed functions so there’s a check for that before we start digging into the parameters and return type.
Code generation
The code generation more or less recurses right down to the type level and then builds up from there. The _rt function is the bedrock which finally resolves concrete types into their appropriate string representations. Once the types are resolved we can build the output of the _func which returns the attach_function bindings for each @Ruby function with a friendly name if specified.
This output from _func is then used to construct the _body generating the rt_init and rt_term definitions on Linux buiilds.
Finally, the output from _body is used in _module where the correct library extensions are determined and calls to rt_init and rt_term are produced if necessary.
Type conversion
The problem is that D types don’t map directly into RubyFFI types in some situations. To handle this there’s a bit of machinery in the _rt() function
alias U = Unqual!T;
return (is(T == cstr) ? "string" :
isPointer!T ? "pointer" :
is(U == ubyte)? "uint8" :
is(U == byte) ? "int8" :
U.stringof);In this code snippet I use T when I want to evaluate the entire type including any storage type qualifiers.
For example cstr is an alias of const(char*) and that maps to a RubyFFI string (assuming you’ve null terminated correctly).
Any pointer type will be mapped to a RubyFFI pointer and you will have to handle the access to that in your code, I will show you examples in the next post in this series.
Finally - when we really don’t want to retain the qualifiers we use U, so something like immutable(int) simply becomes int in the binding output.
Conclusion
There’s still more that can be done with this binding generator to make it better. I don’t like that you have to pipe stderr out to a file to capture the binding output but there’s not really another way to output during CTFE yet (at least not that I’m aware of).
Future improvements could be
- Generating struct layouts
- Generating pointer stubs
- Speeding up the generation process
Check out the rubyffi.d source code
In the next part of this series we will look at more practical applications of using RubyFFI and D.