Page 1 of 1

Is .xml import possible from any software ?

Posted: Tue Dec 27, 2016 2:39 pm
by Clairvaux
Is it possible to import .xml text files generated with any software (say, Word), or is it only possible to import Cinta Notes files previously exported as .xml ?

The first option was my first assumption, however I read a blog post alleging the latter. I tried to import a small .xml file generated by Word 2003. It failed, with a message "Notebook node not found". However, the import menu has both an option for Cinta Notes .db or .xml files, and another just for .xml files.

Re: Is .xml import possible from any software ?

Posted: Tue Dec 27, 2016 3:24 pm
by CintaNotes Developer
I'm afraid that only CN-generated XML files can be imported.
It is simply not possible to import an XML without knowing its structure.

Re: Is .xml import possible from any software ?

Posted: Tue Dec 27, 2016 6:45 pm
by gunars
Alex, is the current XML format used by CN available in case we want to experiment with scripting conversions from other data formats? A while back, I tried a few simple tests with manually created XML files that imported ok, but CN has certainly progressed since then.

Gunars

Re: Is .xml import possible from any software ?

Posted: Tue Dec 27, 2016 6:47 pm
by Clairvaux
Hmmm, thanks. Does not that make Cinta a very closed shop ? It can't import except from itself, and it only exports in .txt, .xml and, in the paid version, .html.

I'm happily using CN right now for non-committal scraping of info from the Web, but what would happen if I wanted to do some critical research based on it, such as writing a thesis, or store years and years of notes in it ? Would that be prudent, given that no software lives for ever (and the database can get corrupted) ?

Re: Is .xml import possible from any software ?

Posted: Wed Dec 28, 2016 6:53 am
by date
I'm happily using CN right now for non-committal scraping of info from the Web, but what would happen if I wanted to do some critical research based on it, such as writing a thesis, or store years and years of notes in it ? Would that be prudent, given that no software lives for ever (and the database can get corrupted) ?
You can backup yourself together with the software. It is about a million times better than storing your notes in some cloud service.

Re: Is .xml import possible from any software ?

Posted: Wed Dec 28, 2016 7:30 am
by CintaNotes Developer
gunars wrote:Alex, is the current XML format used by CN available in case we want to experiment with scripting conversions from other data formats? A while back, I tried a few simple tests with manually created XML files that imported ok, but CN has certainly progressed since then.


The format is not documented, but is fairly simple to read and write. The only notable recent additions were file attachments and rules, both received their own subtags within the "notebook" tag. Also the "note" tag received an optional "attachments" section that references data from the "files" section.

The quickest way to learn the format is by exporting and studying sample notebook data.

Re: Is .xml import possible from any software ?

Posted: Wed Dec 28, 2016 7:33 am
by CintaNotes Developer
Clairvaux wrote:I'm happily using CN right now for non-committal scraping of info from the Web, but what would happen if I wanted to do some critical research based on it, such as writing a thesis, or store years and years of notes in it ? Would that be prudent, given that no software lives for ever (and the database can get corrupted) ?


As already noted, you can pack the portable version of the software together with data.

Also the XML format has a simple structure and it is easy to write a script that would do what you want with it.

What format would you consider to be future-proof? Is there one at all? Even text files don't cut it since there's a lot of metainformation they can't save, like tagging and rules.

Re: Is .xml import possible from any software ?

Posted: Wed Dec 28, 2016 11:13 am
by date
CintaNotes Developer wrote:The format is not documented, but is fairly simple to read and write. The only notable recent additions were file attachments and rules, both received their own subtags within the "notebook" tag. Also the "note" tag received an optional "attachments" section that references data from the "files" section.

Then how do you compress the attachments? I tried to decode the base64-encoded attachment and ended up with an .DMG file (according to TrID) that can't be opened by 7-zip. Very curious about this, could be useful to know for the future.
And, speaking of open formats, what is the exact cipher used with SQLCipher? Is it opened using same password as specified in CN?

Re: Is .xml import possible from any software ?

Posted: Wed Dec 28, 2016 1:53 pm
by Clairvaux
CintaNotes Developer wrote:
As already noted, you can pack the portable version of the software together with data.

Also the XML format has a simple structure and it is easy to write a script that would do what you want with it.

What format would you consider to be future-proof? Is there one at all? Even text files don't cut it since there's a lot of metainformation they can't save, like tagging and rules.


I don't know about the format. I just see that when I try to open the .xml export in the software I use (word processors, browsers), I get nicely structured gibberish which is Chinese to me. I searched for "xml software" or somesuch, but could not find an obvious solution. What does one write a script for xml with ?

I have the installed version. Is it possible to switch to the portable version without losing one's notes ?

Re: Is .xml import possible from any software ?

Posted: Wed Dec 28, 2016 8:40 pm
by date
Clairvaux wrote:I just see that when I try to open the .xml export in the software I use (word processors, browsers), I get nicely structured gibberish which is Chinese to me.

If it is gibberish you are probably looking at the attachments which come at the beginning and take most likely a larger part of the XML file. Scroll to the end (Ctrl-End) in whichever program that can display text files (like XML), and you'll find your notes pretty much human readable, like this:

<note uid="(identifier)" title="your title" created="" modified="" source="" link="" remarks="" tags="" section="0" plainText="1">
<![CDATA[this is your note]]>
</note>

I searched for "xml software" or somesuch, but could not find an obvious solution.

XML is just a format for organized text. Word 2003 saves the page properties, fonts and all the text in XML, and Cintanotes saves the notes in XML. Even the RSS-feed for this forum is in XML. You can open it here: feed.php , right-click and 'View Source' to see what it looks like. But you can't import the feed into CN, just like you can't open a CN XML file in your feed reader.


What does one write a script for xml with ?

If for some reason you want to extract the Cintanotes XML and bend it to something different, for all I know, autohotkey... But you probably don't need to do that, but if you somehow need to in 20 years, you or someone else can get your notes out of the XML in many ways and it's not difficult.

And by the way, the sqlite .db file is an open format as well. About the encrypted .db files and the attachments, I just asked.

I have the installed version. Is it possible to switch to the portable version without losing one's notes ?

Well your notes are in a file that ends in .db. That is the file you need to locate and take good care of, and as long as you know where it is you can open it with any CN. For the installed version, the standard .db is probably somewhere in 'Application Data' somewhere, but you can quickly find with it Windows Search and if that doesn't work then you click Start, Run, and type 'cmd' enter 'cd c:\' enter 'dir/s/b *.db' enter, and have a list of all .db files.


Also with CN, there is no need to create an account first, you have your notes on your own computer. No calling home, no forced updates, no need for an internet connection and all that. So it is far away from being a closed shop. And many advantages over text files. :D

Re: Is .xml import possible from any software ?

Posted: Thu Dec 29, 2016 4:34 pm
by CintaNotes Developer
date wrote:Then how do you compress the attachments? I tried to decode the base64-encoded attachment and ended up with an .DMG file (according to TrID) that can't be opened by 7-zip. Very curious about this, could be useful to know for the future.

Attachments are compressed with zlib, cut into 1MB chunks and base64-encoded.

And, speaking of open formats, what is the exact cipher used with SQLCipher? Is it opened using same password as specified in CN?

Not exactly. An SHA-256 hashing is applied to the password N times, and the result is used as the SQLCipher key.

Re: Is .xml import possible from any software ?

Posted: Thu Dec 29, 2016 4:38 pm
by CintaNotes Developer
I don't know about the format. I just see that when I try to open the .xml export in the software I use (word processors, browsers), I get nicely structured gibberish which is Chinese to me. I searched for "xml software" or somesuch, but could not find an obvious solution. What does one write a script for xml with ?

I guess it would be better to consider the XML that CN produces a prorietory format, since it's not documented. No other program can read it directly. Scripts can be written using any scripting language like Python or Javascript, but it will require some programming skills.

For that matter, it might probably be easier to deal with the .db file directly. It's nothing more than an SQLite database, and there are plenty free tools to work with these, like SQLiteSpy. That's enough for accessing text data, but extracting attachments and text formatting would be much more difficult. So, the main point here: if you REALLY need it, it is possible. The data will be there. But probably not worth your time. Just keep a copy of CintaNotes around.

I have the installed version. Is it possible to switch to the portable version without losing one's notes ?

Sure. Simply copy the contents of %APPDATA%/CintaNotes into the portable CintaNotes folder.

Re: Is .xml import possible from any software ?

Posted: Thu Dec 29, 2016 6:48 pm
by date
CintaNotes Developer wrote:
date wrote:Then how do you compress the attachments? I tried to decode the base64-encoded attachment and ended up with an .DMG file (according to TrID) that can't be opened by 7-zip. Very curious about this, could be useful to know for the future.

Attachments are compressed with zlib, cut into 1MB chunks and base64-encoded.
That doesn't explain why I can't open the result with 7-zip.
But I found some obscure tool that worked, around halfway here: http://forums.majorgeeks.com/index.php? ... ot.223640/

Not exactly. An SHA-256 hashing is applied to the password N times, and the result is used as the SQLCipher key.
So there is no possible way to open an encrypted database without CN lying around...

Re: Is .xml import possible from any software ?

Posted: Sat Dec 31, 2016 11:43 pm
by Clairvaux
Thanks, Alex and Date.

If I understand correctly, CN is only meant to use data produced by CN, and although you can export to view the contents in a sort of forensic way, export can only be used as a type of disaster recovery -- or manual handling of text by copying and pasting.

I've just made a cute little experiment though, on this professional online tagging service by Thomson Reuters : http://www.opencalais.com/opencalais-demo

I exported one Cinta Note in xml format, then uploaded it to Open Calais for tagging. Apart from the automatic tagging function guessing quite well the meaning of the text, which is not relevant to the discussion, it displayed the note's text with the formatting. Web links were active and blue, bold text was bold, indented text was indented as a quote, and text that I had highlighted in my custom yellow in the note was rendered in italics.

None of the metadata was displayed, though (title, tags...).

Which leads me to think there is some potential here, beyond what is officially possible right now.

What is the HTML export mode meant for ? Just viewing in the browser, and nothing else ? Does it show metadata in a properly formatted way ?

Re: Is .xml import possible from any software ?

Posted: Sun Jan 01, 2017 12:49 pm
by date
Clairvaux wrote:If I understand correctly, CN is only meant to use data produced by CN, and although you can export to view the contents in a sort of forensic way, export can only be used as a type of disaster recovery -- or manual handling of text by copying and pasting.

Kind of, but you make it sound like something difficult, but it is very easy to understand.
CN just exports in text, and XML is text that is structured, so it is more versatily conversible.
A common use of XML export is to export part of a database, then import in another database.

I've just made a cute little experiment though, on this professional online tagging service by Thomson Reuters : http://www.opencalais.com/opencalais-demo

That looks like mind-reading software that guesses what the text is about, not directly related to what XML is. (Do you think CN should do more mind-reading :) )

None of the metadata was displayed, though (title, tags...).

I'm not sure but it is my understanding that 'metadata' is part of the data that is usually invisible, but needed for displaying or referencing. Like the uid-tag in the xml.

Which leads me to think there is some potential here, beyond what is officially possible right now.

What do you want to export to or import from? If you say 'this' then someone else says 'that,' and pretty soon the list will be endless. And if they start making it export or import to everything then CN becomes 'data conversion software' instead of 'organising software.' And there is lots of potential for CN in the taking and organising area, it could do more with pictures and clipping, have spellcheck and alarms and reminders and all these kinds of things, and I'd rather have CN evolve in this area, and not in the area of 'data conversion.'
(But if there was a high demand for something in particular, someone could make an external tool for it.)

What is the HTML export mode meant for ? Just viewing in the browser, and nothing else ?

Same as text but prettier and more readable. Perhaps for sending notes to someone else who doesn't have CN, or viewing on your mobile phone if you don't want to use Simplenote.

Does it show metadata in a properly formatted way ?

If by metadata you mean tags and links, then yes, everything except attachments and rules. And no section names, only different colors.

Re: Is .xml import possible from any software ?

Posted: Sun Jan 01, 2017 3:31 pm
by Clairvaux
Date,

Thank you for your information on HTML export. For the rest, I'm not sure I get your drift. You say that the aim of XML is to export to another database, but then you say that there's no need to export anything from Cinta Notes and that we should be happy with the data staying there.

Alex seems to have settled the issue by saying that for all practical purposes, we should consider CN's XML as a proprietary format, because it's undocumented.

I just mentioned a small experiment I did, precisely exporting CN's XML to another "thing" online (is that a database ? I don't know), and showing that something could be made out of it. This shows, I think, that in theory, in the future, at some point, this xml export feature of CN could be applied in practice to something else than re-importing into CN. That's all.

What I'm interested in is expanding the practical uses of the software I have. I use Cinta Notes as a knowledge manager, and the more a knowledge management tool can exchange data with other software, the more useful it is. Pushing the envelope with the tools at your disposal is a natural thing to do : it's what computers are about.

Re: Is .xml import possible from any software ?

Posted: Sun Jan 01, 2017 5:19 pm
by date
Clairvaux wrote:You say that the aim of XML is to export to another database, but then you say that there's no need to export anything from Cinta Notes and that we should be happy with the data staying there.

To export and import to another Cintanotes database. Primarily.

Alex seems to have settled the issue by saying that for all practical purposes, we should consider CN's XML as a proprietary format, because it's undocumented.

It is, but the XML is very readable and it it theoretically not difficult to bend it to something different, because of the nature of XML.

I just mentioned a small experiment I did, precisely exporting CN's XML to another "thing" online (is that a database ? I don't know), and showing that something could be made out of it. This shows, I think, that in theory, in the future, at some point, this xml export feature of CN could be applied in practice to something else than re-importing into CN. That's all.


Yes, exactly. And this is what makes CN a 'not-a-closed-shop.'

What I'm interested in is expanding the practical uses of the software I have. I use Cinta Notes as a knowledge manager, and the more a knowledge management tool can exchange data with other software, the more useful it is. Pushing the envelope with the tools at your disposal is a natural thing to do : it's what computers are about.

Understand, but I think having CN do more with note taking and management should have a higher priority, for several reasons I just mentioned. Can only speak for myself of course...

Re: Is .xml import possible from any software ?

Posted: Tue Jan 10, 2017 11:14 am
by CintaNotes Developer
date wrote:That doesn't explain why I can't open the result with 7-zip.

Does 7zip claim somewhere that it can open zlib's format?

I'm surprised that there's no command line tool for working with zlib format. This is one of the most popular
compression libraries out there.

So there is no possible way to open an encrypted database without CN lying around...

Just use portable CN for temporary decyphering ;)

Re: Is .xml import possible from any software ?

Posted: Tue Jan 10, 2017 11:17 am
by CintaNotes Developer
Clairvaux wrote:What I'm interested in is expanding the practical uses of the software I have. I use Cinta Notes as a knowledge manager, and the more a knowledge management tool can exchange data with other software, the more useful it is. Pushing the envelope with the tools at your disposal is a natural thing to do : it's what computers are about.


I understand your needs very well, however the problem is that there's no standard format for knowledge exchange. Every tool has its own way to classify information, and it is generally not possible to suit all the needs. A simple and readable XML is the closest we can get to interoperability, but you still need to walk the last mile and write the script to convert this XML to the end format you need.