python 3.x - How to enable FTS5 search a string with ".", "_" and "0-9"? -
i have table holding 300k records of strings using alphanumeric, digit, dot, underscore , brackets [].
i use fts5 extension sqlite3 enable fast search on table. how create fts virtual table:
database = sqlite3.connect("mydb.db") db_cursor = database.cursor() db_cursor.execute("create virtual table field_names using fts5 (full_path)") i adding ~300k records using below code in loop:
database.execute("insert field_names(full_path) values (?)", (field_path,)) sample records:
a.extbootrecord.field_db0 a.extbootrecord.field_db1 a.extbootrecord.field_db8 a.extbootrecord.field_db9 a.extbootrecord.field_db10 a.extbootrecord.field_db11 a.extbootrecord.field_db12 a.extbootrecord.field_db15 using following query:
db_cursor.execute("select full_path field_names field_names = '\"%s\"'" % search_phrase) return_list = list() entries = db_cursor.fetchmany(100) while entries: return_list.extend([entry[0] entry in entries]) entries = db_cursor.fetchmany(100) with following search_phrase yields following:
ext: nothingextbootrecord: recordsextbootrecrd.: recordsextbootrecord.fie: nothingextbootrecord.field: recordsextbootrecord.field_db1:a.extbootrecord.field_db1record, expect field_db1, field_db10, field_db11... returned
seems missing fts configuration consume ., _ , 0-9 valid characters part of token.
i tried configurating fts tokenizer using tokenize = \"unicode61 tokenchars '_.'\" in creation statement, no luck.
what missing?
Comments
Post a Comment