Increasing the size of default memory cache makes search spectacularly quicker

Clairvaux
Posts: 65
Joined: Sun Dec 25, 2016 12:43 pm
Contact:

Increasing the size of default memory cache makes search spectacularly quicker

Postby Clairvaux » Mon Apr 01, 2019 4:14 pm

I just solved (partially) an irritating problem I have had since my database grew to a certain size (currently, around 10 000 notes under the same tab, with no attachments).

Search got painfully slow, and especially typing search keywords, as I described in that post : http://cintanotes.com/forum/viewtopic.php?f=4&t=2501. CN would stop responding as soon as a single character was typed, so a search for word would look like : w (wait...) o (wait...), and so on for anything you would type (or delete) in the search field.

I just discovered by chance that increasing the value of sqlite.pragma.cache_size, in the cintanotes.settings file, from the default 4 000 to 8 000, greatly reduced that pause between characters, to the point that it's now tolerable (just). It has not disappeared, but it's manageable.

Increasing it from 8 000 to 16 000 did not produce obvious advantages.

I also changed the sqlite.pragma.synchronous value from NORMAL to OFF. The help advises to set it to NORMAL or FULL if experiencing database corruption, and warns that so doing will slow down CN, since it will wait after each modification till the data is written to disk before returning control to the user. However, on my system, the default was NORMAL.

Previously, my CN occupied around 15 MB of private memory. Now it takes from 70 MB to 130 MB. I have 4 GB RAM, and mechanical hard disks.

Speed increased as a result on other CN operations, too.

While what I wrote in that post about search is still valid, I would suggest the sqlite.pragma.cache_size default value might be too low. Also, the default for sqlite.pragma.synchronous seems inconsistent with what's written in the help. Since those settings are completely hidden, correct default values are critical.
Thomas Lohrum
Posts: 1324
Joined: Tue Mar 08, 2011 11:15 am

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby Thomas Lohrum » Tue Apr 02, 2019 6:08 am

Hi Clairvaux,

thanks a lot for the hint. I increased the size for my database to 8000. My database contains about 7000 notes. It did increase overall performance significantly. Memory usage went up from 41 MB to 48 MB.

Thomas
Kharg
Posts: 7
Joined: Sat Mar 17, 2018 4:40 am
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby Kharg » Tue Apr 02, 2019 7:14 pm

Thanks for the tip.
My database is about 21000 notes, I set the value to 32000 and now it runs perfectly!
User avatar
CintaNotes Developer
Site Admin
Posts: 5001
Joined: Fri Dec 12, 2008 4:45 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby CintaNotes Developer » Mon Apr 08, 2019 4:21 pm

Indeed, increasing the cache size will boost performance a bit, especially on non-SSD disks, in exchange for extra memory usage.
We will probaby change the default value of this parameter in future versions. Thanks!
Alex
DavidCamero
Posts: 1
Joined: Fri Apr 05, 2019 7:05 am
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby DavidCamero » Tue Apr 09, 2019 7:11 am

This tip is awesome and works great for me. But is there a cap for size of memory cache?
User avatar
CintaNotes Developer
Site Admin
Posts: 5001
Joined: Fri Dec 12, 2008 4:45 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby CintaNotes Developer » Wed Apr 10, 2019 10:24 am

David, I'm not sure, but I guess to test this out you'd have to create a notebook that is larger in size that the amount of available memory.
Alex
Clairvaux
Posts: 65
Joined: Sun Dec 25, 2016 12:43 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby Clairvaux » Wed Apr 10, 2019 1:46 pm

What about sqlite.pragma.synchronous ?

I'm a bit at a loss with the automatic / non automatic saving of notes. I left at a point where saving was supposed to be automatic, but I'm not so sure it hasn't been changed with some version. It seems now I have to hit Ctrl + S, if I want to save a new note before it has been closed.

What is the default / recommended setting for sqlite.pragma.synchronous ? What's the difference between the three states ?
User avatar
CintaNotes Developer
Site Admin
Posts: 5001
Joined: Fri Dec 12, 2008 4:45 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby CintaNotes Developer » Thu Apr 11, 2019 7:50 am

Basically, sqlite.pragma.synchronous tells the OS if it's allowed to delay a bit writing the data physically to disk after it has been saved by the code in CN. "FULL" and "NORMAL" are both safe enough, "FULL" being a bit safer and slower. "OFF" is not safe - if your PC suffers from an elecrical blackout just after you saved a note, the note may not be in fact saved.

As you see, there's no connection to saving the note command per se - it's needed anyway. CN prompts to save any changed note when you close it, with the ability to click on "Don't ask again", after that it will save.

Also CN supports autosave. Basically, the edited note is saved to a special place automatically every several seconds. When you explicitly save it (or cancel the changes), the autosaved copy is silently removed. If, however, the autosaved note is still there upon startup, CN assumes there's been a crash and replaces the note with the autosaved version.
Alex
Clairvaux
Posts: 65
Joined: Sun Dec 25, 2016 12:43 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby Clairvaux » Thu Apr 11, 2019 2:00 pm

Thank you. Very informative.

Edit : it might be useful to adjust the help accordingly. It implies OFF is the default, while it seems NORMAL actually is, and your explanation appears to advise it as the default as well :

sqlite.pragma.synchronous
Three possible values: FULL, NORMAL and OFF. If you experience frequent DB corruption (might be relevant for unstable systems and hot-unplugged USB drives), try setting this setting to NORMAL or FULL. CintaNotes will run slower, waiting after each modification till the data is written to disk before returning control to the user.
User avatar
CintaNotes Developer
Site Admin
Posts: 5001
Joined: Fri Dec 12, 2008 4:45 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby CintaNotes Developer » Wed Apr 17, 2019 9:34 am

Indeed the help is a bit misleading. Fixed it, thanks.
Alex
foasolution
Posts: 1
Joined: Thu May 16, 2019 4:48 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby foasolution » Thu May 16, 2019 5:00 pm

Can we set the memory size of cache or it is set automatically.??
User avatar
CintaNotes Developer
Site Admin
Posts: 5001
Joined: Fri Dec 12, 2008 4:45 pm
Contact:

Re: Increasing the size of default memory cache makes search spectacularly quicker

Postby CintaNotes Developer » Sat May 18, 2019 6:02 pm

Roughly, memory taken by cache is proportional to cache_size * page_size

Note also that the page_size setting is used to create new databases, once a database is created its page size can't be changed.
Alex

Return to “CintaNotes Personal Notes Manager”