Page 1 of 1

Increasing the size of default memory cache makes search spectacularly quicker

Posted: Mon Apr 01, 2019 4:14 pm
by Clairvaux
I just solved (partially) an irritating problem I have had since my database grew to a certain size (currently, around 10 000 notes under the same tab, with no attachments).

Search got painfully slow, and especially typing search keywords, as I described in that post : http://cintanotes.com/forum/viewtopic.php?f=4&t=2501. CN would stop responding as soon as a single character was typed, so a search for word would look like : w (wait...) o (wait...), and so on for anything you would type (or delete) in the search field.

I just discovered by chance that increasing the value of sqlite.pragma.cache_size, in the cintanotes.settings file, from the default 4 000 to 8 000, greatly reduced that pause between characters, to the point that it's now tolerable (just). It has not disappeared, but it's manageable.

Increasing it from 8 000 to 16 000 did not produce obvious advantages.

I also changed the sqlite.pragma.synchronous value from NORMAL to OFF. The help advises to set it to NORMAL or FULL if experiencing database corruption, and warns that so doing will slow down CN, since it will wait after each modification till the data is written to disk before returning control to the user. However, on my system, the default was NORMAL.

Previously, my CN occupied around 15 MB of private memory. Now it takes from 70 MB to 130 MB. I have 4 GB RAM, and mechanical hard disks.

Speed increased as a result on other CN operations, too.

While what I wrote in that post about search is still valid, I would suggest the sqlite.pragma.cache_size default value might be too low. Also, the default for sqlite.pragma.synchronous seems inconsistent with what's written in the help. Since those settings are completely hidden, correct default values are critical.

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Tue Apr 02, 2019 6:08 am
by Thomas Lohrum
Hi Clairvaux,

thanks a lot for the hint. I increased the size for my database to 8000. My database contains about 7000 notes. It did increase overall performance significantly. Memory usage went up from 41 MB to 48 MB.

Thomas

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Tue Apr 02, 2019 7:14 pm
by Kharg
Thanks for the tip.
My database is about 21000 notes, I set the value to 32000 and now it runs perfectly!

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Mon Apr 08, 2019 4:21 pm
by CintaNotes Developer
Indeed, increasing the cache size will boost performance a bit, especially on non-SSD disks, in exchange for extra memory usage.
We will probaby change the default value of this parameter in future versions. Thanks!

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Tue Apr 09, 2019 7:11 am
by DavidCamero
This tip is awesome and works great for me. But is there a cap for size of memory cache?

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Wed Apr 10, 2019 10:24 am
by CintaNotes Developer
David, I'm not sure, but I guess to test this out you'd have to create a notebook that is larger in size that the amount of available memory.

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Wed Apr 10, 2019 1:46 pm
by Clairvaux
What about sqlite.pragma.synchronous ?

I'm a bit at a loss with the automatic / non automatic saving of notes. I left at a point where saving was supposed to be automatic, but I'm not so sure it hasn't been changed with some version. It seems now I have to hit Ctrl + S, if I want to save a new note before it has been closed.

What is the default / recommended setting for sqlite.pragma.synchronous ? What's the difference between the three states ?

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Thu Apr 11, 2019 7:50 am
by CintaNotes Developer
Basically, sqlite.pragma.synchronous tells the OS if it's allowed to delay a bit writing the data physically to disk after it has been saved by the code in CN. "FULL" and "NORMAL" are both safe enough, "FULL" being a bit safer and slower. "OFF" is not safe - if your PC suffers from an elecrical blackout just after you saved a note, the note may not be in fact saved.

As you see, there's no connection to saving the note command per se - it's needed anyway. CN prompts to save any changed note when you close it, with the ability to click on "Don't ask again", after that it will save.

Also CN supports autosave. Basically, the edited note is saved to a special place automatically every several seconds. When you explicitly save it (or cancel the changes), the autosaved copy is silently removed. If, however, the autosaved note is still there upon startup, CN assumes there's been a crash and replaces the note with the autosaved version.

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Thu Apr 11, 2019 2:00 pm
by Clairvaux
Thank you. Very informative.

Edit : it might be useful to adjust the help accordingly. It implies OFF is the default, while it seems NORMAL actually is, and your explanation appears to advise it as the default as well :

sqlite.pragma.synchronous
Three possible values: FULL, NORMAL and OFF. If you experience frequent DB corruption (might be relevant for unstable systems and hot-unplugged USB drives), try setting this setting to NORMAL or FULL. CintaNotes will run slower, waiting after each modification till the data is written to disk before returning control to the user.

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Wed Apr 17, 2019 9:34 am
by CintaNotes Developer
Indeed the help is a bit misleading. Fixed it, thanks.

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Thu May 16, 2019 5:00 pm
by foasolution
Can we set the memory size of cache or it is set automatically.??

Re: Increasing the size of default memory cache makes search spectacularly quicker

Posted: Sat May 18, 2019 6:02 pm
by CintaNotes Developer
Roughly, memory taken by cache is proportional to cache_size * page_size

Note also that the page_size setting is used to create new databases, once a database is created its page size can't be changed.