I did UncloudNotes in Feb 2018, and the migrations worked fine then. I just went back and reran the migration from v2 to v4, with iOS 12.2 in the simulator, and it failed with the same error. (This is why it is good to save the data from the various versions for future testing!)
Googling, I found this article, written in Feb 2019, which describes what turns out to be the issue. It has to do with Write Ahead Logging, referred to as WAL, where SQLite is writing new data to a logging file, and only occasionally moving that data back into the main data file. Data in the WAL file no longer gets included in the migration, so it is in the wrong format when the database opens. (I’m not sure why it worked before.)
This is what my data copy of v2 looks like:
Note that the sqlite-wal file is 2.7 MB, and that data is what is not getting migrated.
The solution, from the link above, is to first force the sqlite database to checkpoint, that is to write the wal data into the main data (the first file in the list, shown as 9.7 MB).
I copied the method used at the link, and the supporting functions, and rearranged them a bit to fit in with the UncloudNotes project. It consists of an extension to DataMigrationManager, and then adding a call to forceWALCheckpointingForStore to the CoreDataStack closure at the top, just before the migration gets started:
var stack: CoreDataStack {
let stack = CoreDataStack(modelName: modelName)
if enableMigrations, !store(at: storeURL, isCompatibleWithModel: currentModel) {
forceWALCheckpointingForStore(at: storeURL)
performMigration()
}
return stack
}
The extension listed below creates a temporary persistent store, with the sqlite option “journal_mode”: “DELETE” included. This means the wal file will get written to the main file each time the database is closed (deleting the data from wal). Once it has opened the database, it just closes it again by calling remove, which triggers the checkpoint write.
The result is a successful migration and a database that will open normally again. The resulting data files show that the wal file has been cleared out:
Here is the extension, added at the bottom of the DataMigrationManager file.
//MARK: - DataMigrationManager extension to force WAL checkpoint
extension DataMigrationManager {
func forceWALCheckpointingForStore(at storeURL: URL) {
guard
let metadata = metadataForStoreAtURL(storeURL: storeURL),
let currentModel = compatibleModelForStoreMetadata(metadata)
else { return }
do {
let coordinator = NSPersistentStoreCoordinator(managedObjectModel: currentModel)
let options = [NSSQLitePragmasOption: ["journal_mode": "DELETE"]]
let store = loadPersistentStore(
coordinator: coordinator, at: storeURL, options: options)
try coordinator.remove(store)
} catch let error {
fatalError("failed to force WAL checkpointing, error: \(error)")
}
}
func compatibleModelForStoreMetadata(
_ metadata: [String : Any]) -> NSManagedObjectModel?
{
let mainBundle = Bundle.main
return NSManagedObjectModel.mergedModel(from: [mainBundle], forStoreMetadata: metadata)
}
func loadPersistentStore(
coordinator: NSPersistentStoreCoordinator, at storeURL: URL,
options: [AnyHashable : Any]) -> NSPersistentStore
{
do {
return try coordinator.addPersistentStore(
ofType: NSSQLiteStoreType, configurationName: nil, at: storeURL, options: options)
} catch let error {
fatalError("failed to load persistent store, error: \(error)")
}
}
}
Fun!