Perfect ! You did a great job here. I did a test merge successfully. Does it check for column order ? Several columns don't show up, including the last 3 (Favorite, VisibleInCS and DateAdded) which have changed order since the start of ToneX. Your soft displays Tag_Date whereas ToneX displays DateAdded (blunder on IK Multimedia for this one), so merging can be flawed on this point.

Since I've spent countless hours on these damn databases, if I can be of any help with development, I'll be happy to help !

 

Thanks MaxSxB!

If you double-click an entry in the table, a dialog should show up with all columns in the database, in the main screen I had to reduce the amount of columns. But then copied is each single one column! Currently the tool doesn´t respect each sides database scheme at all, I select one dataset from one side and try to insert it as exactly the same on the other side. I have not list of database fields I check, I read all columns! No changes on the scheme are made from the tool, I leave every database scheme as it is. But in general you can define fields in a database as mandatory or not, latter one then gets default values, this means that different database versions don´t have to lead to errors. While coding this tool, I´ve never came across issues with scheme problems, but I am quite new to ToneX. I checked the least version of the database and I see both fields (Tag_Date, DateAdded) in the table ToneModels and only Tag_Date in table Presets. I don´t know the history about it...

If you want to support: Currently most important is the input what features are relevant. Next is overworking the crappy search. Because I don´t want to change the database scheme, it´s not that easy to get an acceptable search done.

 

What I mean is, does the tool copy entries like column 1 then 2 then 3 then 4 etc. regardless of what they are, or is it like GUID goes to GUID, Tag_Date goes to Tag-Date whatever the order is in the database ?

There might be a problem if sister site database try to merge with legit databases, as there are 2 more colums in legit ones. I'll beta test this and keep you posted

Tag_Date is the date of release on tone.net and DateAdded is the date you downloaded that profile on your library. ToneX displays DateAdded in the Date column, not Tag_Date.

Regarding features, I'd say choosing what columns to display or not could be cool (GUID, the primary key, is a cool column to show)

 

What the tables show is not what the tool copies and the order is therefore irrelevant. For copying it selects the whole dataset again by GUID, containing all fields available in the source database and then builds a INSERT-query (see QueryBuilder.java) for inserting it into destination database. So both databases should have the same scheme. The problem could be, that the source database is newer as the destination database and has fields the destination database doesn´t know, then the copying would probably fail. Solution would be then a update-mechanism for the database-scheme, which would not be rocket-science but would bring in some more complexity.

 

Version 1.0.1 is out, see details here: https://www.codence.de

 

Funny, I have implemented a routine to filter out the GUIDs from showing in the tables! Can you explain what the usecase is with GUIDs?

 

Not much, but several times a month some users reupload profiles they already uploaded in the past. Even with another name. Resulting in duplicates when merging. It would be a way to confirm the error is caused by a duplicate. I might have another reason why but cannot remember it.

I couldn't experiment more yet sadly. I would already have a list of cool features to have, more often easy than not, but I'll think this through before I post in. Anyway, your soft looks very promising

 

GUIDs can be changed as easy as the names of the models. We would have to check the models itself, which is as easy as comparing the GUIDs, because the models itself are stored in a dataset (binary data as (encrypted) base64-encoded ascii-strings).

 

Version 1.1.0 is out, see details here: https://www.codence.de

 

I've seen you posted v1.1.1 as well, and I've been thinking about the whole thing.

First, there is a little bug : when double clicking on an entry to see the whole data, the X button doesn't close the window. Clicking Cancel works though, so not a big deal.

Second (bug I think ?) : I'm pretty sure some minutes ago I could do a right click on an entry to open a dialog box with several options. Now I can't anymore, regardless of the version used. I'll reboot my PC, try it again and edit the post depending on the result. A working dialog box with an option to delete an entry can make your software all-in-one, with no need to use ToneXdbExplorer !
EDIT : Still can't right click after reboot and using a fresh download of latest version. Am I dumb or hallucinating ?

Third : there are 2 more columns needed to properly elevate your soft : Tag_UserName and TargetOrder.
- Tag_UserName because some users are FOS bullcrap needing to be eradicated from any database (I mean, why the hell would you upload Line6 POD captures, deemed the worst sims of all times ?), and some are golden ones that hit bullseye every single time
- TargetOrder, because it shows if a profile is amp, stomp, stomp&amp or complex rig. It is personnaly my 2nd critera of research after ModelCategory

Fourth, regarding the columns order displayed on main window, for an ideal browsing experience I suggest the following :
GUID ; Tag_ModelName ; Tag_ModelCategory ; TargetOrder ; Tag_Description ; Tag_ModelComment ; Tag_AmpName ; Tag_CabName ; Tag_CabMic1 ; Tag_CabMic2 ; Tag_Date
If you want to reduce the number of columns displayed, I think both CabMic ones could be removed without affecting user experience. But having them is still a good thing.

Fifth, if you could add a filtering option for shown columns, you would officially have the best ToneX related software on the whole internet !

Sixth and last : I can't get the database schme thing out of my head. Like I already said, legit libraries have a different scheme than non-legit libraries. ToneX itself doesn't care about the scheme, meaning we can rearrange libraries without consequences. Could we think about an easy non-tideous way to add a button next to Copy All in Tools to match schemes ? I can research and provide the corresponding SQL query to ease your work charge.

Man I'm stoked !

 

Last edited: Jan 3, 2024

For columns order, SQLiteStudio does this set of requests when I want to move them :

Code:

PRAGMA foreign_keys = 0;

CREATE TABLE sqlitestudio_temp_table AS SELECT *
                                          FROM ToneModels;

DROP TABLE ToneModels;

CREATE TABLE ToneModels (
    GUID                VARCHAR (32)  PRIMARY KEY
                                      NOT NULL,
    Version             TINYINT (10)  DEFAULT '1',
    Target              TINYINT (10),
    TargetOrder         MEDIUMTEXT,
    TierEncrypt         VARCHAR (100),
    Tier                TINYINT (10),
    IKGeneratedEncrypt  VARCHAR (100),
    IKGenerated         TINYINT (10),
    Skin                MEDIUMTEXT,
    Instrument          TINYINT (10),
    Copyright           MEDIUMTEXT,
    Model               MEDIUMTEXT,
    CabModel            MEDIUMTEXT,
    Tag_ModelName       VARCHAR (32),
    Tag_UserName        VARCHAR (32),
    Tag_Date            DATE,
    Tag_Keywords        VARCHAR (32),
    Tag_Description     VARCHAR (32),
    Tag_ModelCategory   VARCHAR (32),
    Tag_AmpName         VARCHAR (32),
    Tag_StompName       VARCHAR (32),
    Tag_AmpChannel      VARCHAR (32),
    Tag_ModelComment    VARCHAR (32),
    Tag_CabCategory     VARCHAR (32),
    Tag_CabName         VARCHAR (32),
    Tag_CabMic1         VARCHAR (32),
    Tag_CabMic2         VARCHAR (32),
    Tag_Outboard        VARCHAR (32),
    Tag_CabModelComment VARCHAR (32),
    Favorite            TINYINT (10),
    VisibleInCS         TINYINT (10),
    DateAdded           DATE,
    VisibleInCSEncrypt  VARCHAR (100),
    Factory             TINYINT (10)  DEFAULT '0'
);

INSERT INTO ToneModels (
                           GUID,
                           Version,
                           Target,
                           TargetOrder,
                           TierEncrypt,
                           Tier,
                           IKGeneratedEncrypt,
                           IKGenerated,
                           Skin,
                           Instrument,
                           Copyright,
                           Model,
                           CabModel,
                           Tag_ModelName,
                           Tag_UserName,
                           Tag_Date,
                           Tag_Keywords,
                           Tag_Description,
                           Tag_ModelCategory,
                           Tag_AmpName,
                           Tag_StompName,
                           Tag_AmpChannel,
                           Tag_ModelComment,
                           Tag_CabCategory,
                           Tag_CabName,
                           Tag_CabMic1,
                           Tag_CabMic2,
                           Tag_Outboard,
                           Tag_CabModelComment,
                           Favorite,
                           VisibleInCS,
                           DateAdded,
                           VisibleInCSEncrypt,
                           Factory
                       )
                       SELECT GUID,
                              Version,
                              Target,
                              TargetOrder,
                              TierEncrypt,
                              Tier,
                              IKGeneratedEncrypt,
                              IKGenerated,
                              Skin,
                              Instrument,
                              Copyright,
                              Model,
                              CabModel,
                              Tag_ModelName,
                              Tag_UserName,
                              Tag_Date,
                              Tag_Keywords,
                              Tag_Description,
                              Tag_ModelCategory,
                              Tag_AmpName,
                              Tag_StompName,
                              Tag_AmpChannel,
                              Tag_ModelComment,
                              Tag_CabCategory,
                              Tag_CabName,
                              Tag_CabMic1,
                              Tag_CabMic2,
                              Tag_Outboard,
                              Tag_CabModelComment,
                              Favorite,
                              VisibleInCS,
                              DateAdded,
                              VisibleInCSEncrypt,
                              Factory
                         FROM sqlitestudio_temp_table;

DROP TABLE sqlitestudio_temp_table;

PRAGMA foreign_keys = 1;

This set is executed in less than one second. From what I understand, it proceeds as follows :
- copies data in a new temporary table
- then deletes the original ToneModels table
- then creates a new ToneModels tables with the right scheme
- then copies data from the temp table to the new table
- then deletes the temp table

Considering that I need more than 5mn to add 1000 lines, copy 1000 entries, paste 1000 entries and commit changes, I don't understand how this set of request is so fast. Either SQLiteStudio is far sub-optimal, either the GUI is sloooooow compared to plain text requests !

Please note again that VisibleInCSEncrypt and Factory are only created if the library is opened in a legit instance of ToneX. No problem having them in a non-legit instance though. Since sister site libraries do not include them, it makes sense for me to put them in last, thus avoiding any conflict when copying with your software.

Really, I think implementing a button for these requests is mandatory for your soft to be usable for legit AND non-legit users without further hassle nor use of another software to take care of this problem. It has to be done on both source and destination databases. Maybe you can even include these requests directly in the "Copy selection from top to bottom" function.

 

Many thanks for your input!

First: I am on it, meanwhile you could also use the ESC-key to cancel/close it without saving
Second: You´re hallucinating! There´s never been a right-click context-menu...
Third/Fourth: I am thinking about a configuration where each user can configure his own columns. It´s not only about showing them, I also have to select them from the database. Currently I only show/select a subset to keep the memory consumption low
Fifth: I´ve expanded the search to search in those columns, so you could simply enter your filter into the search field. The fields to be searched could also be added to the configuration done for Third/Fourth
Sixth: Applying a SQL-file/-query to a connected database should be easy. Have you ever tried to cross-copy those databases? Copying from a non-legit database where fields/tables are missing into a full-blown legit database could(!) work ...

 

Last edited: Jan 4, 2024

Seems like a bit of a sledgehammer method to me. Let me think about it, my feeling is that I would prefer to check the scheme and only apply changes where needed ...

 

I still could not experiment further, but I asked ChatGPT and it gave the same method. I specifically asked if there is a way to move columns, it said nope

 

Not sure if it helps but I use the following script to merge dbs. It will Insert by naming the tables so, if there are a different schema in the destination, it's set to the default value, like this. It's bash script, but the same can be done in Java.

Code:

#!/bin/bash

# Check if the correct number of arguments are provided
if [ $# -ne 3 ]; then
    echo "Usage: $0 <database1> <database2> <output_db>"
    exit 1
fi

# Save the arguments in variables
db1="$1"
db2="$2"
cp "$db1" "$3"
output="$3"

# Resolve and format column names
TN_COLUMNS=$(sqlite3 $db2 "pragma table_info(ToneModels)" | cut -d '|' -f 2)
TN_COLUMNS_QUOTES=$(for column in $TN_COLUMNS; do echo "\"$column\","; done)
TN_COLUMNS=$(for column in $TN_COLUMNS; do echo "$column,"; done)

PR_COLUMNS=$(sqlite3 $db2 "pragma table_info(Presets)" | cut -d '|' -f 2)
PR_COLUMNS_QUOTES=$(for column in $PR_COLUMNS; do echo "\"$column\","; done)
PR_COLUMNS=$(for column in $PR_COLUMNS; do echo "$column,"; done)

# Merge the two databases and save the result in the output database
sqlite3 "$output" <<EOF
ATTACH DATABASE '$db2' AS db2;
INSERT OR IGNORE INTO main.ToneModels (${TN_COLUMNS_QUOTES%?}) SELECT ${TN_COLUMNS%?} FROM db2.ToneModels;
INSERT OR IGNORE INTO main.Presets (${PR_COLUMNS_QUOTES%?}) SELECT ${PR_COLUMNS%?} FROM db2.Presets;
DETACH DATABASE db2;
UPDATE main.ToneModels SET VisibleInCS = 1 where VisibleInCs = 0;
EOF

 

And where do you guys the library files from?

 

Sister site, twice a month lately

 

I tried to to cross-copy between a non-legit (source) and a official database (destination) and was successful without a single problem. I don´t get the point why we want to change the scheme of the source database as long as it can be imported successfully. Could you please explain?

 

Well there are 2 cases :
- The first databases posted here on this thread have a different scheme : VisibleInCS, Favorite and DateAdded were not in the same order.
- Legit DBs created by ToneX add VisibleInCSEncrypt and Factory in the middle of these 3 aforementioned columns. Non-legit DBs got rid of these 2 columns. When a non-legit DB is opened with a legit ToneX, they are created automatically after the 3 columns. Thus causing a non-corresponding scheme between legit DBs and non-legit DBs used with legit DBs.

The risk of unmatched schemes are ending up with a non-date entry in DateAdded and a date entry in another column. In the past, I already had Favorite values on DateAdded and vice-versa.

If your Copy alorithm does not take account of the scheme, then there is no problem and I'm grinding gears for nothing lmao. If it does, it can be a problem

 

Let me do further checks tomorrow. Generally I am taking each field existing in a dataset of a table and write them into the new database. It's very generic without any magic, and the order of the fields is irrelevant because I explicitly name every field I insert in the destination database. There are two ways of inserting data into a table: INSERT INTO table VALUES ("val1", "val2). Here the values have to come in the right order as defined in the database scheme. If you use a statement like INSERT INTO table (col2, col1) VALUES ("val2", "val1") you don´t have to respect the order of the scheme. And that's what I am doing with the tool.

I do last checks upcoming weekend and will then release a new version, where I implemented most of the things you mentioned earlier this day!

 

Last edited: Jan 6, 2024