Unsafe Swift: Using Pointers And Interacting With C

[rayfix]

In this tutorial you will learn how to use unsafe Swift to directly access memory through a variety of pointer types.

---

This is a companion discussion topic for the original entry at https://www.raywenderlich.com/780-unsafe-swift-using-pointers-and-interacting-with-c

[aeberbach]

Hey Ray, nice article! I really like this as pointer access in Swift was one of the first things I struggled with in Swift, being an old C programmer I was trying to write some code that would carve out roguelike maps in a block of memory “knowing” that using actual Swift structs per location would be far too inefficient. (Today I think differently.)

What I have a problem with is the “pointee” member… in English the “ee” suffix is something you add to a verb to mean the object of that verb, for example “employEE”. I could see assigning a var to pointee, something with actual storage allocated to it - meaning that the var is the thing pointed to by the pointer. However to assign a value to pointee where there is no object, or storage allocated, seems very wrong - in this case pointee seems like it would be better named “value”. Or simply dereferenced with * as in C.

I had better go start an argument about this on the Swift mailing list!

[atrick]

Fantastic article!

Typed memory must be initialized before use and deinitialized after use. This is done using initialize and deinitialize methods respectively.

Footnote:
That is a fine guideline. But readers of the preceding example will
wonder why they need to deinitialize an Int. Technically they
don’t. It’s ok to skip deinitialization for trivial types – types
that don’t hold heap referenences or require cleanup for any other
reason. Integer and float scalars are obviously trivial. Aggregates of
trivial types are trivial. Enums with only trivial payloads are
trivial.

It’s not so much that raw memory doesn’t need to be deinitialized. The difference is that raw memory can’t be deinitialized because the compiler doesn’t know that type of values that it holds. If the memory in fact holds nontrivial values, then it needs to be deinitialized via a typed pointer to that memory.

Note that loading and storing nontrivial types from raw memory is prohibited.

[rayfix]

Thanks. Glad you enjoyed it!

value is a loaded term in Swift because you could be storing to a value or a reference type. (In this tutorial I mainly stuck with so-called trivial types, which are mostly value types, but there could be reference types too.)

I personally like the term pointee (though it certainly was not my decision) I think it means “the thing being pointed to.” With UnsafePointers, if you are not careful, it certainly could point to hyperspace but that is why they are unsafe.

If you wanted to you could define your own dereference operator. But I wouldn’t recommend it.

prefix operator *
extension UnsafePointer {
    static prefix func *(p: UnsafePointer) -> Pointee {
        return p.pointee
    }
}

extension UnsafeMutablePointer {
    static prefix func *(p: inout UnsafeMutablePointer) -> Pointee {
        return p.pointee
    }
}

func printIt(pointer: UnsafePointer<Int>) {
    print(*pointer)  // prints 111
    print(*(pointer+1)) // prints 222
    print(*(pointer+2))  // prints 333
    print(*(pointer+3))  // prints 4409336256
}

var data = [111,222,333]
printIt(pointer: &data)

Happy hacking!

[rayfix]

Great to have your feedback! There is a link to the definition of trivial types but it doesn’t come until later in the context of rebinding. Readers might miss it. I updated the post to reference your comment so they know that deinit is optional for trivial types. Thank you!

[atrick]

Another little clarification for any readers who like to overthink things:

// Breakin’ the Law… Breakin’ the Law (Undefined behavior)
let typedPointer2 = pointer.bindMemory(to: Bool.self, capacity: count * 2)

This line just rebinds the memory to Bool. It’s a legitimate thing to do. It’s like batch reinterpreting all those bytes as a different type. But typedPointer1 becomes invalid at this point in the program. The undefined behavior comes later when typePointer1 is reused after binding memory to a different type. That’s why it’s safer to use the explicitly scoped withMemoryRebound(to:capacity:) API.

I’m glad you mentioned that it’s illegal to rebind memory in a way that changed trivial types to non-trivial types.

[atrick]

One final observation:

By using the inout & marker it is taking your compression_stream and turning it into a UnsafeMutablePointer<compression_stream> automatically. (You could have also just passed streamPointer and not needed this special conversion.)

Yes, but people should realize this line copies the stream object (which seems like extra work):

var stream = streamPointer.pointee

Readers can also easily make the mistake of thinking that stream and streamPointer.pointee are the same object. When in fact, the memory pointed to by streamPointer is never even initialized in your example.

Unfortunately, you do need to dynamically allocate compression_stream since you’re not initializing it in Swift code. But to avoid deferencing streamPointer everywhere, you can do something like this:

  func initStream(stream: inout compression_stream) {...}
  func destroyStream(stream: inout compression_stream) {...}

  // 2: initialize the stream
  initStream(stream: &streamPointer.pointee)

  ...

  destroyStream(stream: &streamPointer.pointee)

[carlhung]

yes, i was suffering by this part. i thought pointee returns something that points to this memory until reading you comment. thank you.

[rayfix]

Thanks for the clarification. Will fix this.

[rayfix]

One person’s “elegant solution” is another person’s hack I guess.

What would you think if I reworded it this way?

// create a stream, work around compression_stream doesn't have a default Swift initializer
var stream: compression_stream
do {
    var streamPointer = UnsafeMutablePointer<compression_stream>.allocate(capacity: 1)
    stream = streamPointer.pointee  // copy it out, even though still uninitialized
    streamPointer.deallocate(capacity: 1)
 }

Worse? Those nested functions aren’t making me happy.

To avoid all of the streamPointer.pointee dereferencing, either way you still need to make a copy so I think both approaches are the same amount of work. It is really too bad I couldn’t say:

var stream = compression_stream(operation, algorithm)

and be done with it. Maybe that is worth a radar?

[rayfix]

What do you think of the version I put there in the comment?

[aeberbach]

It still bothers me! If pointee is “the thing being pointed to” that’s OK - but it’s really clashing with my C idea of a pointer being the address of storage. You can’t take the address of “4”, because it isn’t a thing, just a value. The storage where “4” is stored is somewhere of course but we’re not pointing to that.

[jtoronto34]

This is a great article but I’m having a little trouble here with a specific API. I’m trying to create a CTRubyAnnotation (for furigana) and here’s what it takes according to the function reference:

func CTRubyAnnotationCreate(_ alignment: CTRubyAlignment, _ overhang: CTRubyOverhang, _ sizeFactor: CGFloat, _ text: UnsafeMutablePointer<Unmanaged<CFString>>!) -> CTRubyAnnotation

So it has the double-whammy of needing an Unmanaged CFString wrapped in an UnsafeMutablePointer?

The best example I can find of usage is here, but it seems that even he himself doesn’t know how it works, or even if it does at all. I can’t get it to work.

       let furigana: UnsafeMutablePointer<Unmanaged<CFString>>! = UnsafeMutablePointer<Unmanaged<CFString>>.allocate(capacity:4)
        furigana[0] = Unmanaged.passUnretained(rubytext as CFString)
        let annotation: CTRubyAnnotation = CTRubyAnnotationCreate(CTRubyAlignment.auto, CTRubyOverhang.auto, 0.5, furigana)

I’m getting a EXC_BAD_ACCESS crash at the CTRubyAnnotationCreate line, with no console output.

Does it need to be initialized? Or is it automatically initialized when he assigns the value to [0]? Why allocate 4?

助けてくれ！よろしくお願いします！

[rayfix]

Hi! Not familiar with the API at all. :] It seems like it essentially wants is a totally unmanaged array of strings. So I am thinking something like this:

var strings = ["Hello, playground"]

withUnsafeBytes(of: &strings) { arrayPtr in
  // At this point you have a raw buffer pointer into the contiguous memory
  let ptr = arrayPtr.baseAddress?.bindMemory(to: Unmanaged<CFString>.self, capacity: str.count)
  type(of: ptr)  // should have the correct type at this point
}

How does that work?

[jtoronto34]

@rayfix

var strings = [“Hello, playground”]

withUnsafeBytes(of: &strings) { arrayPtr in
// At this point you have a raw buffer pointer into the contiguous memory
let ptr = arrayPtr.baseAddress?.bindMemory(to: Unmanaged<CFString>.self, capacity: str.count)
type(of: ptr) // should have the correct type at this point
}

Thank’s for replying!

Getting the following hilarious compiler error:

 error: 'count' is unavailable: there is no universally good answer, see the documentation comment for discussion

I don’t see where str is declared. I’m thinking you meant the strings array, so if I change it to that, and have the following line:

let annotation: CTRubyAnnotation = CTRubyAnnotationCreate(CTRubyAlignment.auto, CTRubyOverhang.auto, 0.5, ptr)

The compiler shows an error for ptr and says undeclared identifier 'ptr', did you mean 'str'?

I’m thinking there’s something to str that I’m not understanding.

However if I let it auto-fix to str, it then says cannot convert value of type 'String' to expected argument type 'UnsafeMutablePointer<Unmanaged<CFString>>!'

There’s zero documentation for this API from Apple as well.

Actually here’s a dump of the console output when running the code exactly how you pasted it

Playground execution failed: error: WKReaderViewPlayground.playground:60:100: error: 'count' is unavailable: there is no universally good answer, see the documentation comment for discussion
            let ptr = arrayPtr.baseAddress?.bindMemory(to: Unmanaged<CFString>.self, capacity: str.count)
                                                                                                   ^~~~~

error: WKReaderViewPlayground.playground:60:100: error: 'count' is unavailable: there is no universally good answer, see the documentation comment for discussion
            let ptr = arrayPtr.baseAddress?.bindMemory(to: Unmanaged<CFString>.self, capacity: str.count)
                                                                                                   ^~~~~

error: WKReaderViewPlayground.playground:72:114: error: use of unresolved identifier 'ptr'
       let annotation: CTRubyAnnotation = CTRubyAnnotationCreate(CTRubyAlignment.auto, CTRubyOverhang.auto, 0.5, ptr)
                                                                                                                 ^~~

WKReaderViewPlayground.playground:1:5: note: did you mean 'str'?
var str = "Hello, playground"
    ^

What’s it talking about var str = "Hello, playground"? Nowhere is it declared that way.

[rayfix]

Sorry! I meant strings.count (Aside: str.count is complaining because str is presumably declared a string and you need to get a character view… str.characters.count… but that is another subject).

I meant ptr. ptr cannot escape the scope of the unsafe bytes… it guarantees it stays allocated and bound within that scope. So it would be correct to do:

var strings = ["cool"]

var annotation: CTRubyAnnotation!
withUnsafeMutableBytes(of: &strings) { arrayPtr in
  let ptr = arrayPtr.baseAddress?.bindMemory(to: Unmanaged<CFString>.self, capacity: strings.count)
  annotation = CTRubyAnnotationCreate(CTRubyAlignment.auto, CTRubyOverhang.auto, 0.5, ptr)
}

The types work out but, alas, it crashes! CTRubyAnnotationCreate must have specific semantic requirements!

If you command click on the CTRubyAnnotation it takes you to the header documentation which is better than the generated documentation. I don’t know what this means yet but it says:

    @param      text
                An array of CFStringRef, indexed by CTRubyPosition. Supply NULL for any unused positions.

If you look at the objective C annotations it looks like it expects EXACTLY four entries. Now the sample code that your link provided starts to make some sense.

https://developer.apple.com/reference/coretext/1510191-ctrubyannotationcreate?language=objc

So I wrote this:

var hiragana = "test" as CFString
var furigana: UnsafeMutablePointer<CFTypeRef> = UnsafeMutablePointer<CFTypeRef>.allocate(capacity: 4)
defer {
  furigana.deallocate(capacity: 4) // don't forget to clean up
}
furigana.initialize(to: kCFNull, count: 4)
furigana[0] = hiragana as CFTypeRef

var ruby: CTRubyAnnotation!
furigana.withMemoryRebound(to: Unmanaged<CFString>.self, capacity: 4) { ptr in
  ruby = CTRubyAnnotationCreate(CTRubyAlignment.auto, CTRubyOverhang.auto, 0.5, ptr)
}

This compiles and doesn’t crash. I have no idea how to use a ruby object though. You will have to take it from there. I would be interested to know if it works!

[jtoronto34]

Thanks!! It compiles but I am getting a crash at this line

let framesetter = CTFramesetterCreateWithAttributedString(attribString)

with the following message -[NSNull length]: unrecognized selector sent to instance 0xe2e858
I’m creating the attributed string using the ruby object like this:

attribString.addAttribute(kCTRubyAnnotationAttributeName as String, value: ruby, range:furiganarange)

Still working on figuring it out. I’ll let you know how it works out!

This is incredibly complicated stuff and I’m hoping to learn how it ALL works by the time I’m through this stumble.

[jtoronto34]

FWIW I was able to get through this by creating the attributed string in an Objective-C class. I’m thinking there might be an issue with the Swift implementation as it seems like a very low priority API. Either that or I have no idea what I’m doing, which I at least know to be true.

Thanks you for the help though.

[rayfix]

Glad you got it working! FWIW, if you can show the working Obj-C version I might be able to write the equivalent Swift version. It would be nice to put this complicated stuff in a nice Swifty wrapper. :]

[jtoronto34]

This is the Objective C method I used.

+(CFAttributedStringRef)generateString:(NSString*)inputString
                 WithRubyforRanges:(NSDictionary*)range
                   WithOrientation:(GlyphOrientation)orientation{

//Just for test purposes only
  NSString *testFurigana = @"テスト";
NSRange furiganaRange = NSMakeRange(0, 3);
NSDictionary *furigana = [NSDictionary dictionaryWithObject:testFurigana forKey:[NSValue valueWithRange:furiganaRange]];

CFAttributedStringRef attrString = CFAttributedStringCreate(kCFAllocatorDefault, (CFStringRef)inputString, nil);

CFMutableAttributedStringRef mutableAttrString = CFAttributedStringCreateMutableCopy(kCFAllocatorDefault, CFAttributedStringGetLength(attrString), attrString);




    CFAttributedStringBeginEditing(mutableAttrString);
    for (NSValue *value in furigana.keyEnumerator) {
        NSRange range=value.rangeValue;
        NSString *string=[furigana objectForKey:value];
        CFStringRef furigana[kCTRubyPositionCount] = {(__bridge CFStringRef)string, NULL, NULL, NULL};
        CTRubyAnnotationRef rubyRef = CTRubyAnnotationCreate(kCTRubyAlignmentAuto, kCTRubyOverhangNone, 0.5, furigana);
        CFRange r=CFRangeMake(range.location, range.length);
        CFAttributedStringSetAttribute(mutableAttrString, r, kCTRubyAnnotationAttributeName, rubyRef);
        CFRelease(rubyRef);
    

}


CFAttributedStringEndEditing(mutableAttrString);
CFAttributedStringRef rubyString=CFAttributedStringCreateCopy(NULL, mutableAttrString);
CFRelease(mutableAttrString);




return rubyString;
}

And calling it on the swift side.

 let attribString = WKRubyAnnotatedStringGenerator.generate(inputText, withRubyforRanges:nil, with: .horizontalOrientation)
        
 let framesetter = CTFramesetterCreateWithAttributedString((attribString?.takeRetainedValue())!)