python 3.x - How to enable FTS5 search a string with ".", "_" and "0-9"? -
i have table holding 300k records of strings using alphanumeric, digit, dot, underscore , brackets [].
i use fts5 extension sqlite3 enable fast search on table. how create fts virtual table:
database = sqlite3.connect("mydb.db") db_cursor = database.cursor() db_cursor.execute("create virtual table field_names using fts5 (full_path)")
i adding ~300k records using below code in loop:
database.execute("insert field_names(full_path) values (?)", (field_path,))
sample records:
a.extbootrecord.field_db0 a.extbootrecord.field_db1 a.extbootrecord.field_db8 a.extbootrecord.field_db9 a.extbootrecord.field_db10 a.extbootrecord.field_db11 a.extbootrecord.field_db12 a.extbootrecord.field_db15
using following query:
db_cursor.execute("select full_path field_names field_names = '\"%s\"'" % search_phrase) return_list = list() entries = db_cursor.fetchmany(100) while entries: return_list.extend([entry[0] entry in entries]) entries = db_cursor.fetchmany(100)
with following search_phrase
yields following:
ext
: nothingextbootrecord
: recordsextbootrecrd.
: recordsextbootrecord.fie
: nothingextbootrecord.field
: recordsextbootrecord.field_db1
:a.extbootrecord.field_db1
record, expect field_db1, field_db10, field_db11... returned
seems missing fts configuration consume .
, _
, 0-9
valid characters part of token.
i tried configurating fts tokenizer using tokenize = \"unicode61 tokenchars '_.'\"
in creation statement, no luck.
what missing?
Comments
Post a Comment