changelog¶
Note
This is the new changelog, only the most recent builds. For all versions, see the old changelog.
Version 695¶
seekbar¶
- to relieve the 'oh my click did something else because I didn't notice the tiny gap below the seekbar in fullscreen mode' issue, if the current media has a seekbar and it is wide but not quite tall enough for the window--think a vid like 1920x1078 in fullscreen--hydrus now drops that media the couple pixels down so the seekbar is flush with the bottom edge whenever 'centering' that media. you can change the pixel gap threshold for this (default 20) or turn it off entirely, under
options->media playback - this 'move down' mode should work well in UI scales larger than 100%, but you will still sometimes get videos with a little gap at the bottom when the height of the media matches the height of the window; there are no easy and excellent technical solutions for this right now, but if you want me to add an option to always drop it a virtual pixel and thus always leave a little gap at the top, I can do it
- fixed an issue where in fullscreen borderless in some UI scales, the seekbar does not pop up to full size if the mouse is at the bottom of the screen
- UI scales greater than 100% should draw the border around the scanbar a little better. not perfect yet
more memory-efficient duplicate work¶
- a user did some great memory and CPU profiling on larger potential duplicate pair filtering and found bloat due to massed primitive object duplication. I have worked on this and potential duplicate filtering should now use at least 67% less memory, and in complicated situations 90%+
- specifically, the core search atom of this system is now completely deduplicated in all cases, so any new search is pointing to the same memory store, and only containers need be recreated
- further, I reworked a rich index used to search these guys at the object level to now use these new objects, compressing used data a bit more, particularly in search spaces of millions of rows
- when you do hit large search spaces, I think this compressed tech will also save a bunch of CPU when updating and re-initialising searches. let me know how it goes
- also fixed a bug in 'group mode' potential duplicate pair search that wasted a little CPU and in very very rare cases might deliver a pair with a file outside the group
- also, the 'clear search context' signal, which happens when a user purges all potential dupes from the db, is better about clearing out data here
exe manager¶
- the new and still-under-testing 'get metadata for a single file' job type under
options->external programsnow shows a 'result parsing' box with the same basic UI as in 'manage parsers'. write up a json or html parser for whatever you are getting back on stdout or stderr, and the newly updated test panel will show your raw and parsed output! - next step here is to migrate the legacy 'file lookup scripts' tag suggestions box to be a 'go ask external thing for tag suggestions' guy, and then you'll be able to pipe whatever ai model or script you can think of that eats a file reference and generates tags right into 'manage tags'
misc¶
- fixed some crazy, probably truncated animated gifs that were raising errors when Pillow tried to iterate through their num_frames
patreon¶
- nothing bad is happening, but my personal situation is changing a little and I will be closing my Patreon page later this month. the links and pages in the help are removed today to prep for this
- I expect to keep working on hydrus just like normal, with regular schedule
- thank you to everyone who supported over the years. I really appreciate it!
environment stuff¶
- for advanced source users, the setup_venv.py now tentatively supports python 3.15, with the proviso that you need to select advanced/test on everything and you will not get lz4 (some memory savings) or jxl (the filetype) support yet
- removed the 'openssl' version line from
help->about. this wraps the system version, not anything the python venv actually controls, and isn't particularly helpful. the old-date-despite-being-updated is sometimes confusing/worrying for users, too. EDIT: ironically, we ended up looking at openssl versions despite it being non-python this week when we went through some dll fun. I'll bring this back, and other versions we can see, in a new about tab
boring stuff¶
- fixed a double-layout-set on the main gui that was raising a warning in PyQt6
- a routine that streams blocks of data from external executable calls (for ffmpeg video rendering and silent audio checks) now does a CPU-efficient sleep (this continues some CPU-thrashing code cleanup work I started a few months ago)
- the fast event processor 'job schedulers' are optimised a little and have less per-loop overhead
- when thread slots are saturated (this happens in the downloader system when there are lots of downloaders waiting to work) downloader jobs now back off on reattempts, slowly moving from 10 second to 180 second waits
- I wrenched the parser edit ui's edit and test panels apart--very ugly decoupling, and there is more to do--so I could embed the edit UI in the executable callable panel
- the edit parser panel now has a nicer Qt update signal on any changes
- a bunch of list handling tools (mass unions, iterator reshapers, some auto-throttling chunkers) have nice type hints, so all callers know the shape of what they are getting back based on what they are giving
- bit of misc linting
boring mainloop refactoring¶
- I did a KISS and merge pass on my manager daemons, such as the things that run file maintenance, sibling/parent calc, and subscriptions. a bunch of timing/waiting code is more solid, but I may need to revisit this to check on some edge cases, particularly around a remaining ugly knot of when new jobs come in while a daemon is cleaning up what it thinks is the final job. I'll keep pushing on this
- the parent class of mainloops now handles the pausing between loops; every subclass now receives desired work period and returns 'still work to do' status, and the parent makes waiting decisions based on this, in a unified way
- forced wait times are also now handled by the parent class
- the separate idle/work waits are merged and wakes are handled instead by current work state logic
- the top mainloop now handles most 'check if shutting down' calls
- all nine mainloop managers now do CPU-efficient sleeps when not working, which should reduce some thrashing in busy clients
Version 694¶
misc¶
- added 'confirm add-undelete tags' to the cog icon on 'manage tags'. if you have been annoyed at the 'confirm your action' popups in the new manage tags dialog when you type in a tag that was previously deleted, please uncheck this and let me know how it feels
- added 'allowed actions for taglist double-click:' to this menu too. if you don't ever want to see a 'clear deletion record' option, or whatever, you can now turn things on and off here too
- fixed the 'clear trash' routine to delete more than one old file per run. this was the lagging logic patched last week, now improved back to what it was originally intended to do
- also made the clear 'trash routine' wait on 'a file was recently deleted' safety delay more intelligently (it doesn't work if a file was deleted in the past sixty seconds, as a grace period for undeletes of accidental giant deletes), and, lmao, it no longer sends this signal itself when it works (blocking itself a bunch when there's a lot of work to do)
- when you drag-and-drop URLs on the client, and the application source bundles non-URL data in the 'urls' metadata slot (we have evidence of one less-used browser putting the browser tab text in with the url), the hydrus drag-and-drop catcher now filters the urls from the non-urls nicely. if any urls were in there, it silently prints info to the log; if nothing was an url, it puts it in a popup for you to see
- fixed some login cookie handling when using the
curl_cffitest. there is more work to do with the curl_cffi test, so please bear with me - I fixed some issues with the 'children' tab of tag autocomplete, which looks up children of tag parents in the associated taglist, when the file domain is set to multiple file domains or 'all known files'
exe manager¶
- the new Windows 'hide terminal' tech test is now default ON for all users. the test appeared to go fine, so I am making it real. you shouldn't notice any difference, but if you are a Windows user and you suddenly get command line popups every time you import a file or do 'open externally', hit up
options->external programsto switch it off, and let me know!
v693a hotfix¶
- I did a hotfix on Saturday to address some update stuff--
- in the 'future build' integration, the pyproject.toml accidentally did not get the intended
pyopenssl,numpy, orpyside6updates, meaning the builds missed out. it was just a copy/paste transcription failure by me. this is fixed and I'll make a script to do this automatically in future - fixed a bitrot issue with the v687-to-v688 update code
- fixed a typo in deserialisation when updating older metadata routers from v2 to v3 (would hit old import/export folders with sidecars that hadn't been edited in about 18 months)
- as a side thing here, I now have a nicer test in my weekly build workflow that tests my older db updates better. I'm going to keep an older db and test current code to catch bitrot issues like the above. unfortunately, my best rich test db is currently v693, so I'm going to pin it at that version and let it get older every week until it is 50 weeks old and we have full coverage of all the updates I try to support. so, going forward I guarantee I will test anything after v693 every week
- I've made a plan for a serialisable update manager too and will be working on that in the future. we'll fix these legacy objects sitting around on the db for years without a resave
security¶
- I continued the recent security work. my deserialisation pipeline now has a single clean deserialisation context to hold the various trust information. it is easier to pass this through each stage, and this thing will be easier to extend in future
- in doing this, I was able to remove a hack that was suppressing some trusted whitelist warning logging, so whitelist warning logging is now true positive in a couple of places it was false negative before. I already fixed a couple small instances of this, although tbh I was surprised that we had so little report of legacy cruft in the local store; it was mostly just a couple of legacy 'current file service is this' on older page objects
- also generically cleaned some deserialisation unit tests and special deserialisation situations
- there's more I want to do here, including what I hope will be a big deserialisation typedef solution, so I'll keep pushing
boring stuff¶
- added
help->debug->threads and processes->wake mainloopsfor various debug testing - cleared out the 12 or so 'calling non-callable' linting issues with nicer typedefs and some stub callable juggling
- cleaned up all Callable type definitions across the program and updated some old deprecated typing stuff
- fixed some error handling for malformed objects in page session load--it was not recovering properly and raising a new error
- added unit tests for the new cookie serialisation and deserialisation for requests
- added unit tests for the new cookie serialisation and deserialisation for curl_cffi
- fixed an issue with the 'networking' unit test suite when it was run on its own;
tldextractwas not initialised properly in this case, and when it tried to load its suffix list from disk to init its cache inside the first test, the mock http wrapper gonked the load
Version 693¶
security¶
- if you missed it, AI discovered an important security flaw in hydrus last week. if you are running v691 or earlier you should update to v692 or later, such as today's release. check here for more details: https://github.com/hydrusnetwork/hydrus/releases/tag/v692
- I am now starting a project of improving the rigour of hydrus I/O security to shrink the attack surface and reduce the potential severity of future discoveries like this. today's release brings a top-to-bottom overhaul of deserialisation security with a whitelist applied to all untrusted ingresses
- when the hydrus server takes a POST request, it now applies a whitelist of which serialisable objects it is willing to load, based on the request type. if there is extra stuff, it does 400 Bad Request
- conversely, when a client receives responses from the hydrus server, it now equally whitelists the serialised response and throws an error if anything doesn't match the whitelist
- in the client UI, when you paste a bitmap or drop a png onto a list that knows how to import/export, it now applies a similar whitelist before loading; if it is expecting URL Classes, it will recognise anything that isn't an URL class before it tries to load anything and cancel out
- if an object load is from a trusted source (e.g. you are loading your options object from the database), the whitelist does not apply strictly but the log will write about anything that violates it. we'll see how much cruft we have hanging around from old forgotten things. if you get some messages like
An object of type 26 ("Serialisable List") loaded when the expected serialisable types are: {127}! You can probably ignore this, but if there are a bunch of these, hydev would like to know., please pass them on - the hydrus serialisation system will no longer deserialise an object that comes from a future version of the client, under any circumstances. we had a hacky path here previously to permit some 'run the client on old software and see if it muddles through the errors' situations, but I would rather be KISS on this question going forward
misc¶
- fixed an issue introduced in v689 that was causing increments of 0.5s of wasted CPU work when the trash was cleared for old files. some users have reported unusual freezes that come and go; please let me know how things feel now. thanks to the reporting user who found this with AI
- hydrus services now take longer breaks (about four hours) when there are connection problems, and similarly when there is any generic problem. previously, a couple of request paths would not generate a delay and they'd try again ~fifteen minutes later. I've also made hydrus requests cancel earlier in all situations--a couple of slow maintenance paths would run the normal cancel/retry system, but this stuff is mostly hidden from UI and it wasn't helping to have quick retry
- removed some ancient two year old server update code that would no longer be fired anyway because of the bitrot guard
- improved shutdown code to more promptly cancel ongoing network jobs. if a manager (e.g. 'synchronise repositories' manager) is waiting on a network job to clear but there's a connection retry etc... holding things up, this is now insta-cancelled and shutdown no longer waits
- the url class parameter edit panel now makes the default-value string processor available when the 'ephemeral token' checkbox is not set on. there is also a textbox to show the output of the string processing on the current example text
boring security worklist¶
- added an object type whitelist to all deserialisation code and a trusted/untrusted environment bool; if untrusted (e.g. pasting from clipboard, receiving from a server, POST request to server), the whitelist is adhered to strictly and exceptions raised on violations; if trusted (e.g. loading from local database store), we load whatever comes and accept-but-log legacy cruft that exceeds the whitelist
- converted default object loading to use the whitelist in all cases (just a couple calls)
- converted 'noneable' object loading to use the whitelist in all cases (just three calls)
- converted 'networkbytes' object loading to use the whitelist in all cases (15 calls)
- converted 'fromstring' object loading to use the whitelist in all cases (48 calls)
- converted the core 'serialisabletuple' object loading to use the whitelist in all cases (~295 calls aieeeee)
- propagated the trusted/untrusted environment to the serialisable update routines too
- updated serverside POST request processing to obey whitelists particular to each command
- updated client 'Service' request response parsing to obey a similar per-command whitelist
- the 'what is this?' debug panel now separates the payload ingestion from actual deserialisation, showing a preview of the json content before you parse it. there are also two buttons, for trusted and untrusted parse
- added unit tests for the new deserialisation whitelisting
- added a unit test for the new future-object deserialisation error
- added unit tests for the serverside whitelisting
- misc cleanup all around here
more boring cleanup¶
- removed the legacy subscription object and migration code and import support in the subs dialog. the transition was years ago, so we can drop it
- similarly removed the legacy bandwidth manager object and migration code
- amongst the type filtering work this week, discovered that legacy url classes could (harmlessly) insert the incorrect sort of string conversion tool in their parameter definitions, and fixed it
- Lain now skips erroneous import object reports more efficiently. previously she would give an error window for each unexpected object; now only simple page parsing formulae are even parsed and she'll just give one error and silently skip over them after that
- the checkbox list that Lain provides, and the same list used across the program, now makes a better expandable layout when it has more than 20 items in it. previously it janked out and wouldn't grow vertically properly
- extended some lists that accept import/export of hydrus serialisable objects to forgive objects in the same class and give a nicer error. 'import url class' list will eat gugs, ngugs, and such, and say 'hey, not them m8' rather than going nuts. you can drop a bunch of mixed downloader objects on one of these sub-dialogs and it'll pull what it can
- fixed some thumbnail error handling and 'file report mode' reporting, which missed something in the transition from hashes to media_results
future build¶
- This release commits the changes tested with the recent future test build that I did after making v691 last week, which went well
- the summary for this update is: most python libraries are brought up to latest, and ffmpeg, and the builder, pyinstaller, too
- there are no special instructions for the update. update as normal
- since library versions have been bumped, users who run from source will be encouraged to rebuild their venv on update this week. this is particularly important in the current rapid-update climate
- The changes are--
- python updates--
PySide6moved from6.10.3to6.11.2opencv-python-headlessmoved from4.13.0.92to4.14.0.94pillow-heifmoved from1.4.0to1.8.1cbormoved from6.1.3to6.1.5chardetmoved from7.4.3to7.7.0cryptographymoved from49.0.0to50.0.2lxmlfrom6.1.1to6.1.3numpymoved from2.4.6to2.5.4pyopensslmoved from26.3.0to26.4.0- new library--
curl_cffiis now included by default, to help testing in built environments, at0.16.3- build--
pyinstallermoved from6.18to6.22.3- test versions--
PySide6moved from6.11.1to6.12.0- misc--
- ffmpeg updated from
8.1.2to9.0.2on Windows - ffmpeg updated from
2026-06-30to2026-10-21on Linux - updated the tldextract domain list from
2026-07-15to2026-10-20 - updated github actions in workflow scripts to latest stable versions
Version 692¶
security hotfix¶
- An AI scanner has found a remote code execution bug in the hydrus client. This release fixes the issue. All users should update
- the risk occurred when importing a user-made object like a downloader or when talking to a hydrus server like the PTR
- EDIT: I originally thought the Client API was vulnerable, but additional testing has shown it was not.
- The hydrus server was not vulnerable
- After investigating the 5,000+ publicly shared downloaders on the user-run repo and 27,000+ PTR update files, I did not find any use of the exploit and am confident that there was no widespread use of the flaw. This is the time to update, in case the problem becomes real in the near future.
- to help propagate the notice and make a hard split with unpatched clients, the hydrus network version has been bumped (to 21). Any client that tries to communicate with an updated server (which includes the PTR, which has just been updated) will get a message that they need to update.
- The Client API also has a version bump, up to 96
Version 691¶
exe manager¶
- the 'this can live for a very long time' checkbox is renamed and now detaches the process better on Windows and POSIX. the long-time child process should no longer pipe stdout/stderr to our terminal
- the
textcheckbox is removed from 'local process call' edit panels; this will now be inferred from any parsable result settings, and with a specific encoding - the
hide terminalcheckbox is removed from 'local process call' edit panels. this was only affecting Windows and has few debug uses, so the behaviour is now always on - if you are a Windows user, the 'external programs' options panel has a new 'TEST' checkbox to try out some cleaner 'hide terminal' tech. if you are an advanced user, please check out its tooltip and let me know how you get on
- in the first step of getting arbitrary 'tag suggestions' working, you can now set an executable call type of 'get metadata for a single file' type. this exposes a new setting at the bottom of the 'local process call' panel where you can specify whether the metadata arrives on
stdoutorstderr, and in which encoding, if notutf-8. there's no parsing yet (it'll use the same UI as the downloader system), so it will always say it parsed nothing, but the test panel is hooked up and we are now reading basic responses from external tag calls. if you are interested in this stuff, feel free to wire up a test situation
session log-off detection¶
- I wrote a nicer hook for the signal that your OS sends to programs when the user is about to log off. hydrus has never been great about catching these events and sometimes has a non-clean exit when the computer shuts down. I am not 100% confident in my implementation yet, so it is hidden behind a new
TEST: Try new log-off detection for clean exitunderoptions>gui. - added
help->debug->tests->do session sign-off (fake)to simulate this event firing - if the test is on and it happens, your log will especially state "doing fast session shutdown" when it triggers. the program hides and saves without showing the splash
- I'm worried about whether some users session managers fire the event at a time other than logoff, e.g. every fifteen minutes for some extraneous checkpointing that is misfiring, so if you are interested in cleaner program exit on log-off, please switch this test on and let me know how it goes. does it save nicely when you log off, and does the program ever now close randomly?
misc¶
- fixed an issue where the QtMediaPlayer would render flag-rotated videos in a too-small area
- the command palette no longer gives 'label' results (menu items with text but no function) when searching the main menu or media menu
- I juggled some of
help->debugaround, including adding a 'threads and subprocesses' submenu - added a 'review subprocesses' window to this new submenu. it shows the current subprocess snapshot with basic launch command, pid, and running/return code, and buttons to do reaping maintenance, and terminate (+kill on POSIX). as we do more executable manager stuff, we're running into needing a better window on this stuff, so this is a first draft. I think we'll want some more stuff like start time and timeout
boring stuff¶
- I did a bunch of boring cleanup and infrastructure expansion in the exe manager to prep for fetching tags. there's new enums, a nicer response object to handle responses, and the test panel is ready to talk about file/metadata responses
- local process calls now have 'parsable' and 'importable' sourcing data so they can pull a metadata document (with an encoding) or (in future) a raw importable file from a pipe or (in future) a temp_path input parameter
- executable definitions now have a page parser to parse metadata documents in prep for tag suggestions, but there's no UI for it yet
- moved
Switch to main window when creating new file search page from media viewerfromoptions->guitooptions->media viewer - brought
Pillowfrom12.3.0to13.0.0for a security update
Version 690¶
more treeview improvements¶
- the user who has been working on the treeview experiment (under
options->gui pagesif you want to try it out!) has sent in more improvements (#2101)-- - shift-double-click on a page entry in the treeview now does a page rename
- the treeview page entries can now catch file drag and drops!
- the cog icon has a new 'style/display' submenu with the alternate-row-colour setting and a new hide/show focus rectangle option
- some bottom-label/page-name desync issues are fixed
- fix when a page is first activated inside a notebook after selection moves from a different parent notebook, the parent notebook node would be highlighted instead of the activated page
- fix the 'expand all' isnt enabled when i have a pages of pages inside another, and collapse the inner one (and the inverse situation)
- fix some other small selection/activation bugs, clean up and consolidate some redundant code
no more shutdown maintenance¶
- I have completely removed the 'do maintenance on shutdown' system. this system was old, creaky, inefficient, occasionally confusing, and had to jump through several unusual hoops to still work. it used to do a bunch of stuff, but I had whittled it down, over years, to just database analysis and repo processing work. removing it completely satisfies a long-term KISS plan
- going forward, I will continue to push on maintenance work working in small increments in convenient invisible ways. ideally, most users should never care or have to bother with maintenance. I may add a 'yeah go nuts do work now' button for a convenient 'just do idle work now' mode in future, but for now, the plumbing behind shutdown is a bit simpler and a shutdown should now always be as prompt as possible
- the related settings under
options->maintenance and processingare removed, and the shortcut that did 'shutdown and force maintenance' now just does a normal shutdown
manage tags¶
- the manage tags dialog has more options on what happens when you enter tags. delete confirmations only take one 'yes' click, but it also interrupts you more when you are about to do something with no easy undo
- the 'confirm remove/petition tags on explicit delete actions' cog icon setting, which used to only apply on hitting the delete key on the taglist or the 'remove tags' button is now 'confirm remove/petition tags' and applies to all potential removes. instead of making a second yes/no dialog, it forces the action choice dialog to show even if there is only one available action (the delete/petition)
- the action confirmation now appears even if there is only one action to do if that action is an undelete, clear deletion record, rescind pend, or rescind petitioned call
- by default, the autocomplete paste button now only does add/pend actions (so if you paste tags that are already in the list, it will not offer to delete them or rescind pends or whatever), but you can allow it to do delete/petition, resind pend/petition, and clear deletion record if you like under a new cog icon submenu
- by default, autocomplete tag entry can trigger any action except clear deletion record (add, delete, pend, petition, rescind pend, rescind petition), but you can configure it under a similar submenu
- double-clicking the list to 'activate' it can now trigger any action (add, delete. clear deletion record, pend, petition, rescind pend, rescind petition). adding a tag from the tag suggestion boxes is the same (sans 'clear deletion record'), but since those lists filter themselves by tags already in the list, It'd be tricky to pull off one of the 'removes'
- hitting the delete key on a list selection or hitting 'remove tags' button can now trigger delete, petition, rescind pend, or clear delete record outcomes
- this retires the old 'allow remove/petition result on tag input for already existing tag', which was always a muddy fudge that I could never remember the precise meaning of
- when you enter tags, there are more dialog labels to cover different situations. previously it would erroneously say 'of the 1 files being managed, some have that tag, but not all of them do' when showing the new add/clear deleted record choice, but now it handles stuff like that properly. further, it mentions if an 'add' action involves an undelete
related tags¶
- related tags works better in some clever ways; you don't have to do anything
- you can now customise whether 'related tags' starts with "just for my files" and "siblings and parents" on/off when the manage tags dialog boots. there's a cog icon on the related tags panel, and complementary checkboxes added to
options->tag suggestions->related tags - the related tags search system does some smarter sibling lookups when it weights search and result tags, now weighting by the ideal tag even when you search not in the sibling/parent space. if you don't want to receive any 'series' tags in your results, it will now stop giving you unnamespaced suggestions that sibling to a series tag. same deal for its searching tech
- I have made significant improvements in how the 'related tags' list is filtered of tags that already exist in the list, so that it should be better about not suggesting a storage tag when a tag already in the list shares the same ideal tag or already includes the suggested via a parent implication. there are still a couple small holes here, but the improvement was enough that I have removed all the ugly and annoying filtering at the db level, saving some lookup time and making it so if you repeal an already-pending (tag repos) or already-current (local tags) tag, the 'related tags' list is probably going to spontaneously list it, since it already found that guy, knows it is important, and was actually hiding it already. it just makes for a more reactive list that you don't need to refresh so much if you remove something by mistake
import options¶
- the 'import options' button-with-menu-arrow widget you see on every importer now allows you to edit the default import options (essentially just opening the
options->import optionspanel on its own) from the menu. you can also open it from the edit panel itself with a new 'edit defaults' button. if you change the defaults, the panel should update on ok to show any new actual options inheritance - fixed the 'favourites' list in import options panel not sorting on init
- fixed an error when trying to delete one favourite
couple of mpv things¶
- updated the default_mpv.conf with formatting, some opening explanation blurb, and consolidated the info in the experimental confs into it as examples for the advanced user to check out. updated the label in the options help too; ask a chatbot works well for navigating the excellent-but-monstrous docs here: https://mpv.io/manual/master/
- mpv
[error]s ofYour platform doesn't support hardware accelerated Xare now ignored; previously they'd spam to log but the file would render fine
boring cleanup and little fixes¶
- fixed
database->db maintenance->get tables using definitions, which was failing due to a missing subclass method definition in one of the duplicate-file db modules - added a safety catch for file import or gallery download jobs so they always always get a status set, even if an error handler is supposed to but then fails; if nothing is set by an error handler, the job is set as error. we had one instance of this recently that had a looping file import job that was failing to set some sort of skip/ignore/error status because a backing daemon had shut down early
- split the existing 'shutdown' exceptions into 'daemon' vs 'program' shutdown, so catchers can better handle the two situations
- moved the client thumbnail cache to its own file,
ClientCacheThumbnails, and wrote some new thumbnail sizing intention infrastructure - moved all the hydrus core thumbnail code to its own file
HydrusImageThumbnails - moved the filetype determination code out to its own file
HydrusFileTypeDetermination
Version 689¶
all misc this week¶
- with the tests complete, everyone is moved to the new 'GraphicsView' thumbnail rendering system. the old thumbnail grid rendering code and related thumb cache and other objects are deleted
- for local file services, you can now delete a deleted tag record in manage tags. if you are looking at a deleted tags (i.e. one with an X count, like
skirt (X1)), then actioning it (double-click etc..) now gives you the option to either undelete it or clear its deletion record. the 'remove? yes/no' dialog that you may have set to appear when you hit the delete key will now say whether you are about to remove or purge your selection - the daemon that deletes overflowing/old files from the trash (when you give the trash a max size or age under
options->files and trash) is migrated from the old daemon type to the newer. it uses fewer database queries to work, has smarter and lower-latency work/sleep timings that cap out after a half-second of dynamically flexible work, no longer fights with the user if they happen to undelete the oldest trashed files while it is working, avoids work for the sixty seconds after you delete a file from anywhere, and deletes up to the satisfaction of the rules and no further (previously, it would work in batches and cut the trash down to way under the size/age the rules asked for) - a 'is this file in use by another process?' test on to-be-imported files no longer attempts the 'rename file to itself' method in non-Windows. this test was false-positive failing on some POSIX systems with advanced permission setups and had no true-positive outcome
- the 'is this file in use by another process?' test now checks in with
flockin non-Windows systems. it'll try to open a read on the file and grab an exclusive lock; if either fail, it says no. if this flock test causes permission issues in your storage setup, it should still work while spitting info to log; you can turn it off completely with a new checkbox inoptions->files and trash - fixed disappearing system tray icons on 'hide to system tray' on OSes with a Window Manager that needs a full QWidget for the system tray icon. thanks to the reporting user for figuring this out! (issue #2091)
- fixed the 'move left/right one tab/service page' and 'move to previous/next media' shortcut actions not being listed as choosable action types in the autocomplete shortcut set! (issue #2087)
Version 688¶
run external program on import¶
- when importing files, there is a new 'external programs import options'. it allows you to run a 'send single file' call every time a file imports with a result of 'successful (and new)', 'successful (already in db)', or both. if you are building a secondary database of your hydrus files, let's say some special duplicate or search data, you can now auto-populate this guy with your new imports
- when the import options panels are in simple mode, this new options type appears in global, local hard drive import, import folders, and specific import options. I expect most people who use this will want it for either a specific import folder or for everything
exe manager misc¶
- I added a first draft of an 'executable manager' help page to https://hydrusnetwork.github.io/hydrus/external_programs.html . the options page now links to this
- the 'open externally (single file)' and 'open URL (single URL)' executable types are renamed to 'send single file' and 'send single URL'
- the 'default programs' options page is renamed back to 'open externally'
- if you import an local process call with any of: exe path longer than 256 characters; more than 16 parameters; or more than 1024 characters of parameters, then the import routine tells you about the weirdness, showing what is weird, and asks if you are yes/no ok to add it
import options¶
- the
options->import optionspanel now shows your favourites/presets in a new list. this is the stuff normally tucked into the 'star' button, just exposed better. it is all synced with the star button, so changes show up immediately - the box-panels holding the three lists are now also collapsible, if you need more space
- added some simple texts to the boxes and tweaked some labels here. 'favourites' are now generally called 'favourites/presets'
- fixed the default/url class 'paste custom' menu option to actually boot the customise panel, rather than just doing a paste-merge
- the 'Pasted!' micro-notifications in the options panel no longer appear when the paste action is cancelled
misc¶
- added
--non_interactive_updatelaunch switch, which makes the client auto-choose the recommended default choice if the update routine wants to ask the user a yes/no question. if you do headless/absent updates, let me know if this is sufficient - the splash screen now prefers to use an svg, and I have drawn an svg for it to use. it looks like the old png. feel free to create your own
db_dir/static/hydrus_splash.svg(or .png) file and that will be used instead. it renders at 144x196, the size of the old png, but maybe we can adjust a little now we have the tech - fixed a small blank box that was appearing in some stylesheets in the 'external call' box of the new exe manager UI
- fixed thumbnail redraw in the new thumbnail rendering tech when you change stylesheets or flip darkmode or change blurhash rendering
- fixed thumbnail redraw in the new thumbnail rendering tech when you change thumb border or margin
- yes/no dialogs (and some other 'quick' dialogs) now max out at ~48 lines of text height. they get a scrolling panel and everything, but if these guys want to dump a gigantic list or error traceback or something, they'll stop being a giant column
- all file imports will now assign 'ignored' status to a file that is 'already in db' but for which the file filtering options would deny (let's say you have an import folder set up to not get any pngs). previously, these files were set as 'already in db', before the file filtering checks had a chance to make a judgment, and since the file was therefore considered 'successful', additional metadata like tags could be applied even though the file would have been ignored otherwise. it will now be set as 'ignored' and no metadata added
- if you have multiple since/before time predicates of a particular type in a search (e.g. 'imported since two weeks ago' and 'imported since one week ago'), the search is now careful to select the most restrictive of those, satisfying both. previously it pseudorandomly selected one to use (issue #2089)
- fixed the 'x files were not in client' error label when you try to copy some non-sha256 hashes and not all of them can be found. it was counting wrong before and just saying the total selection size
- fixed right-clicking on files with unknown filetype (the new open-with executable stuff wasn't handling it correct)
- if the deferred physical delete system encounters a file with seemingly no file info, the system now stops deferred physical delete for that boot and gives the user a note that they should regen their local hashes cache. this symptom seems to be a knock-on from a desynced hash cache
boring stuff¶
- refactored some 'manage options' stuff so it is less coupled to the dialog
- the favourite tags list is now only refreshed across your autocomplete dropdowns if it actually changes on an options ok
- some custom-colour widgets now only redraw themselves if darkmode actually flips on options ok
- fixed a mermaid chart in the 'virtual memory in linux' help
Version 687¶
misc¶
- the popup toaster is now very anti-taking-focus. if you navigating the thumbs via keyboard while popups bounce around doing their thing, particularly subs, they should now be less likely to steal focus from what you were doing. even clicking buttons shouldn't do it. this is mostly an experiment, so let me know how it works out
- updated
mkdocs-materialfrom6.7.1to6.7.7due to a dependbot alert about an XSS issue with the search box
simpler menubar¶
- in an effort to make the main gui's top-level menubar menus thinner and easier to mouse through, a machete has been applied. long labels are reworked or banished, 'manage x...' is now generally just 'x...', and some 'x and y' are now 'x/y', and some big weird things are pushed down to a submenu. Some tooltips are altered to help. I apologise for the jolt to muscle memory here, but I've been meaning to do this for ages and I'm trying to pack a bunch into one bigger frustration than continually reworking it in dribs and drabs
- one thing I'm not totally happy with is the 'pending' menu. putting the count of pending stuff at the top of the submenu keeps it tucked out of the way, but if you find you are clicking that and not 'commit' two weeks from now, I'll need to re-do it
menubar boring details¶
- under the 'file' menu, I clipped the labels a bit
- the 'undo' menu is fine
- under the 'pages' menu, I pushed the page count and weight measure to a 'weight' flyout submenu, the 'sidebar and preview panels' submenu is now just 'sidebar', and the 'new x page' labels are clipped
- under the 'database' menu, all the long backup gubbins is put into a 'backup' submenu, 'move media files' is renamed to 'locations', and 'file viewing statistics' is renamed to 'clear'
- under the 'network' menu, 'downloader components' are folded into 'downloaders' and the 'logins' legacy label is pushed to the submenu. also, an ancient ancient and hacky tumblr GDPR click-through patch is removed
- under the 'services' menu, we just have 'review', 'edit', and 'administrate'. this is another "manage" to "edit" change, which is nomenclature borne from a stupid old technical difference that doesn't matter to the end user and which I am slowly changing. also this weird 'import update files' job is pushed to an 'advanced' submenu
- under the 'tags' menu, everything is cut to the bone and 'manage where siblings and parents apply' is pushed down to an 'advanced' submenu
- under the 'pending' menu, the count is now pushed to the flyout submenu, making the top menu just your service names. should make it both thinner and less bouncy as things are pending/uploading in the background
- under the 'help' menu, I clipped a couple things
- I think I updated all the help docs to point to the new labels, but let me know if you see something I missed
executable manager¶
- the executable manager is now real. check it out under the 'external programs' options page. it supports 'open single file' and 'open url' pipelines to start, and like other hydrus things the objects can be exported and shared between users in a bunch of ways
- the 'default programs' options page uses the new executable manager. your existing launch paths will be converted to the new system. feel free to rename them. if you are feeling brave, poke around and make your own callable
- when you do 'open externally' on a file or URL, it all works on the new system. this includes stuff like hyperlinks. the call is cleaner, more secure, and has nicer failure reporting
- added a 'show PATH' button to the exe manager's process call edit panel. it throws up the PATH as hydrus sees it with a bit of blurb
exe manager boring stuff¶
- on init or update, an exe manager is created and populated with defaults or a migration of your existing string launch paths
- the exe manager now tracks dirty/clean status and is plugged into the normal manager save checker
- the existing 'open url/file' options are converted to the new calls, and the new exe manager is populated with executables for the old launch paths
- all the 'default programs' UI is converted to the new 'select which exe' system. it provides live choices from the current the 'external programs' page
- hooked up some extra safety code to make sure the exe manager provides the fallback default OS call when there is nothing else set, and to remove defunct items on dialog ok and such
- process calls in the executable manager system now clean their parameters on init, deserialisation, or simple dialog editing. newlines are removed, leading or trailing whitespace is stripped, and multiple whitespace is collapsed to single
- wrote some migration code for the legacy string templates to the new executable callable objects
- cleaned up the whole 'open url/file' pipeline of all sorts of old cruft, and updated it to work on media results rather than media singles (part of a long-time cleanup job). similarly deleted a bunch of old code
- wrote more safety code to inject and fall back to the 'open with default OS call' for 'open file/url' calls that have no options set up for whatever reason
- fixed some edge case harmless errors when hitting 'open file/url' in the media viewer in the moment it is closing
- all the open file/url calls now catch the new exceptions and report in nice info popup dialogs when things go wrong
- fixed some tucked away 'open this url' menu entries (stuff like the regex help link) that were using the wrong 'launch url' call in last week's multi-browser-launch-options refactor
- refactored some 'populate exe manager with defaults' code to share the same call
- refactored the executable manager action calls out of the manager to their own file and moved some 'open url with default browser' calls in there too
- refactored the 'open help docs here' to better tapdance around the new 'open url' pipeline
- wrote up a slender gui-executable-actions wrapper module for all 'open url/file' commands so if there is error, there is always an UI context upon which to throw up an information error message dialog. there is better testing and error reporting when trying to open a non-local file externally
Version 686¶
misc¶
- the 'give child windows a tool flag' test setting went well, and I am switching everyone to use this mode as default now. if you have trouble with 'review services' or similar, please hit
options->guiand hit the newBUGFIX: Set child windows as non-tool flagged - the
force that hitting Enter/Return on radio button lists triggers a dialog okcheckbox is now set to true by default, and everyone is switched over this week. same as before, the test went well. if you prefer it the other way, please hit upoptions->guiand switch it back - the content summary strings in the duplicate auto-resolution thumbnail pair lists have some formatting improvements: the 'add mappings a, add mappings b, add mappings c...' stuff is now 'add mappings: a, b, c'; different service lines are now newline-separated; and the thumbnail rows are a little taller by minimum to typically fit this text in better
- decided to finally remove the long-time disabled 'potential pairs' line from Mr Bones. the newer duplicates page with its fragmentary and cancellable search handles all this better, and since the potential pairs space exists only for local files rather than Mr Bones's ability to look into deleted files, it is best just not to bring this number back here. He now only handles total dupes and alternates made
- improved the maths behind the alternates counts in Mr Bones. it also works a bit faster. the 'x alternate groups' number is now filtered correctly according to the current search context, and some interesting edge cases are consciously handled so a search for 'system:file relationships - number of alternates = 0' produces the correct 0/0 results. there's a couple of ways of looking at these numbers and I've taken the most conservative to occasionally undercount but never overcount
new thumbnail grid rendering tech¶
- we've been testing out a new, cleverer way of rendering the thumbnail grid. it works and looks basically exactly like the old system for now but will support new layouts and resizable thumbs in future. we are happy with the test, so I am switching everyone to use it today. ideally, you do not notice any differences. if you suddenly do have trouble with your thumbs, hit up
options->thumbnails, switch off the new rendering mode, restart the client, and let me know - I made the 'which file to select when you press an arrow/page key' work more humanly in the new grid; now, if your current focus file is removed and you then press a movement key, the 'ghost selection' media is now selected, re-initialising your position. previously, it would navigate from the ghost to a neighbour, which always felt weird. also, the ghost is now the next non-removed file (used to be previous), unless the removees stretch to the end of the list, in which case the previous is chosen. this feels a bit more natural and catches some odd cases since previously it worked on selections rather than removees
- this is more controversial and may mess with muscle memory, but I have also made it so a shift-select does not move the "navigate from here" ghost. this ghost now sticks to the current focus. if this drives you crazy, there is a new checkbox under
options->thumbnailsto switch back to moving the ghost to the last file you hit to alter the shift-selection. if you understand what this means, you are a true patrician and I personally recommend mapping ctrl and shift to your mouse. also, I may start rendering the ghost as a dotted line or something just like you'd get in a file explorer or something - as part of this, I fixed an issue with the new tech where it could have trouble finding a new media to select after certain removal of non-selected items, for instance after zero-selection archive/delete filtering or a background file delete
- fixed the enable-reactivity of the ctrl- and shift-selection checkboxes under
options->thumbnails - I fixed up some various other focus items and 'these were selected in the current shift-select' variables that were not properly cleansing themselves after a media remove event
- the new graphics view thumbgrid test can now process an interactive 'entry dialog' content update shortcut
- fixed a trace error with the interactive 'entry dialog' command when nothing is selected in the thumbgrid
new ways to open files and urls externally¶
- in prep for the exe manager taking over, the options for 'default programs' that handles web browser calls and 'open externally' for different filetypes are updated
- the web browser setting is now a list of calls, so you can set up alternate web browsers or profiles if you like. if you have multiple, the top one is selected for quick/default url-openings and in the media 'urls' menu you now get multiple submenus to choose which one you want
- the file 'open externally' has two changes--now you can set the default 'open externally' call for "all files" and "all images/videos/whatever" as well as for individual filetypes. I hope this makes it simpler to just switch from x image viewer to y for everything while still allowing something special for avif or whatever. your existing options are going to be eaten up and converted to the simplest suitable umbrella solution
- and, secondly, each of those entries now support multiple calls, so you can say 'open images with x program or y' and then, like with the urls, the top item is the default for quick actions but otherwise your media 'open externally' menu is going to dynamically expand into multiple options if you set that up
- the string program cals are all going to be converted to the exe manager in a week or two. feel free to add new paths now, or wait for that, at which point you'll be selecting from a dropdown here
potential pair discovery scheduling logic¶
- I cleaned up a bunch of edge-case logic around how files are set to be searched for potential pairs. an interesting bug (#2086) revealed that previously-deleted files were not being scheduled for re-entry into the potential pairs search system, and when I looked into it, this revealed some more messy logic. I've cleaned it up and now blank square files are 'find similar files' searchable
- first off, when a previously deleted file is re-imported, it is now scheduled for potential-pair discovery as expected
- further, the determinant of whether a file should be searched for potentiar-pair discovery is now strictly that it has non-blank phashes. files with blank or near-blank phashes are now entered into the search tree, but they are not scheduled for potential pair discovery (blank files produce many false positive pairs, so we do not want them)
- the 'ensure file is in similar files search system' job is now careful to only consider that question, and it no longer triggers any pixel or phash regens
- files that are discovered to have no useful phashes are deregistered from the system correctly. previously, some pixel hash and re-import events would trigger an accidental listing or not trigger a proper delisting; now the logic is KISS
- all files that currently have no phashes (blank files and some weird non-renderable things) are scheduled for a phash regen on update, which will make 'system:similar to' work for them
- identical blank files (we had an example of two pure white comic pages with the same resolution, once) that share the same pixel hash will no longer naturally appear in the potential pair search
- pasting a blank or near-blank image into the 'system:similar files' panel also now produces phashes in the box. it is now possible to search for blank and blank-like images with this system. if you are interested, the blank phash is
8000000000000000. paste it in once the maintenance has caught up here and brace yourself for a rollercoaster ride - the names and descriptions of the 'check for membership in...' and 'regenerate perceptual hashes' file maintenance jobs are clarified
boring stuff¶
- fixed an issue where non-advanced users saw the 'external programs (TESTING)' page in the options lol
- cleaned up the application processing code's responses and formalised that a shortcut will be caught if it matches an entry, not only if it matches and the command produced an arbitrary result. if you say 'copy bitmap' when no file is selected, the shortcut is swallowed there; it won't go "ok nothing happened" and see if anything higher matches your shortcut
- a user reported an interesting statistical bug in the duplicate pairs fragmentary search; where it does a confidence interval check, in one case it had more searched items than the entire search space. it seems this was a race condition. I have wrapped all the code in proper locking, added nicer guards before the math, and tidied up some redundant code along the way
- misc linting and typing, blah