vb.net - Improving performance when Marshaling to a complex structure -

this follow question .net file random access recoard locking

i have been able make reading , marshalling work ok performance lot slower using fileget().

there few articles around e.g. https://msdn.microsoft.com/en-us/library/ff647812.aspx

however have use non-blittable entities unfortunately , don't provide working examples on how make performance gains.

examples of current code being used:

<structlayout(layoutkind.sequential, _            charset:=runtime.interopservices.charset.ansi, pack:=1)> structure salbchcx 'sal555     <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xj1s char()   'h/d       <marshalas(unmanagedtype.byvalarray, sizeconst:=8)>     <vbfixedstring(8)> public xj2s char()   '     <marshalas(unmanagedtype.byvalarray, sizeconst:=8)>     <vbfixedstring(8)> public xj3s char()   '     <marshalas(unmanagedtype.r4, sizeconst:=4)>     public xj4 single     public xj5 single     <marshalas(unmanagedtype.r8, sizeconst:=8)>     public xj6d double     <marshalas(unmanagedtype.r4, sizeconst:=4)>     public xj7 single     <marshalas(unmanagedtype.byvalarray, sizeconst:=61)>     <vbfixedstring(61)> public xj8s char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xj8bs char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xj8cs char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xj8ds char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xj8es char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xj8fs char()     <marshalas(unmanagedtype.i4, sizeconst:=4)>      public xj8al integer      <marshalas(unmanagedtype.r4, sizeconst:=4)>     public xj9 single     <marshalas(unmanagedtype.byvalarray, sizeconst:=11)>     <vbfixedstring(11)> public xjas char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=30)>     <vbfixedstring(30)> public xjaas char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=9)>     <vbfixedstring(9)> public xjbs char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=35)>     <vbfixedstring(35)> public xjbbs char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=5)>     <vbfixedstring(5)> public xjcs char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=7)>     <vbfixedstring(7)> public xjds char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=8)>     <vbfixedstring(8)> public xjdas char()      <marshalas(unmanagedtype.r4, sizeconst:=4)>      public xje single     <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xjfs char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=6)>     <vbfixedstring(6)> public xjgs char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=30)>     <vbfixedstring(30)> public xjhs char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=30)>     <vbfixedstring(30)> public xjis char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=30)>     <vbfixedstring(30)> public xjjs char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=30)>     <vbfixedstring(30)> public xjks char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=35)>     <vbfixedstring(35)> public xjkks char()     <marshalas(unmanagedtype.r4, sizeconst:=4)>     public xjl single      <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xjms char()     <marshalas(unmanagedtype.byvalarray, sizeconst:=2)>     <vbfixedstring(2)> public xjns char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=70)>     <vbfixedstring(70)> public xjos char()      <marshalas(unmanagedtype.r4, sizeconst:=4)>     public xjp single      public xjq single      public xjr single      <marshalas(unmanagedtype.byvalarray, sizeconst:=1)>     <vbfixedstring(1)> public xjss char()      <marshalas(unmanagedtype.byvalarray, sizeconst:=128)>     <vbfixedstring(128)> public xjts char() end structure  ....  public sal555 salbchcx   .... dim f5 new filestream(couc, filemode.open, _ fileaccess.readwrite, fileshare.readwrite, marshal.sizeof(sal555))   x = 1 10         getbatchrec(x, f5) 'populates sal555 next x ....   public sub getbatchrec(recnumber integer, file filestream)      dim b() byte     redim b(marshal.sizeof(sal555) - 1)     file.seek((recnumber - 1) * 600, seekorigin.begin) 'marshal.size(sal555)     file.read(b, 0, b.length)     dim h = gchandle.alloc(b, gchandletype.pinned)     sal555 = marshal.ptrtostructure(of salbchcx)(h.addrofpinnedobject())     h.free()  end sub 

we using byvalarray rather byvaltstr byvaltstr did not capture last char of string due expecting ending return char. copy char() strings using sal5.xjis = new string(sal555.xjis) again takes processing around 2ms do.

the main 2 parts of code seem taking of time following.

file.seek((recnumber - 1) * 600, seekorigin.begin) 'marshal.size(sal555) sal555 = marshal.ptrtostructure(of salbchcx)(h.addrofpinnedobject()) 

each operation takes around 10ms , using fileget() uses around 8ms taking around twice slow before copy char() strings.

i hopping had better idea of how read/marshal data complex structure bit more efficient.

any suggestions welcome, richard

fyi: going ignore answers related "stop using flat files" not option present :)

after extensive performance testing discovered solution of marshaling faster vb6 ported fileget().

the misleading thing development environment taking age run through loops when running compiled exe or visual studio inbuilt performance profiler performance of marshalling was better fileget() , not hitting locked records prematurely, win/win!!

learn new every day!

thanks, richard.