Please familiarise yourself with the forum, including policy on feature requests, rules & guidelines

Delurg: A (wip) Deluge File Browser... In the Browser!

8
HailaHaila FinlandPosts: 16

Hey all,

So I finally also got to the point of too many samples and synths and songs and everything is a mess. As a lazy person, I decided to spend a lot of time avoiding a little bit of work and started tapping away on a little tool I call the Delugr.

Killer feature: nothing to install! Just point to your deluge root folder and done.

Access here: delugr.haila.fi
Open source here: Gitlab repo

It's not done or even super useful, but I'm already getting some fun insights from just browsing the most used instruments! Maybe someone else might too, so I put it up with automated builds. Feel free to drop bug reports and feedback, or better yet just fork the thing and send some PRs. I would much rather work on the UI than endless XML parsing :expressionless:

Here's a few screen grabs for convenience:




Post edited by Haila on

Comments

  • 0
    jensgjensg Berlin, GermanyBeta Tester Posts: 297

    Does it work on an iPad and Chrome?

  • 0
    HailaHaila FinlandPosts: 16

    @jensg said:
    Does it work on an iPad and Chrome?

    iOS seems to have "partial support" since a few months ago for the file API. So... maybe? Firefox users seem to be completely out of luck right now. :smiley:

    https://caniuse.com/native-filesystem-api

    Mobile browsers are known to be a royal pain to debug. Stack traces etc. much appreciated if you find that it doesn't work!

  • 2
    HeptagenHeptagen Posts: 277

    The fact that this just works in the browser is absolutely amazing! Well done!
    And seeing which sounds I use the most is very helpful. I sense quite a lot of potential in this project. I'd love to see you keeping it going and developing it further! 💪

  • 0
    Bay_MudBay_Mud OntarioBeta Tester Posts: 123

    This is cool! I love the idea of seeing most used.

  • 0
    HailaHaila FinlandPosts: 16

    Thanks for the kind words! I'll keep tapping away with the goal of getting sample stats and broken sample detection going. Need to spend some time figuring out how to detect and parse the various firmware formats first :smile:

    I'll update the post here when I feel like something newsworthy has happened.

  • 0
    rezareza los angelesModerator, Beta Tester Posts: 603

    sweet, might be valuable to check if the synth preset used in songs are the same as the saved synth preset or if they've been adjusted in anyway. might save some confusion!

  • 1
    HailaHaila FinlandPosts: 16

    @reza said:
    sweet, might be valuable to check if the synth preset used in songs are the same as the saved synth preset or if they've been adjusted in anyway. might save some confusion!

    I think that makes a lot of sense. It implies some way to open a synth instance (not just the preset) and visualize it in a meaningful-enough way to detect and highlight key differences between that and another instance. A puzzle for a rainy day!

  • 1
    tomtobblestoptomtobblestop Portland, Maine, USABeta Tester Posts: 16

    I just stumbled across this and wanted to give some kudos. I keep a version controlled local directory of my Deluge SD card and I love that with your site I can point at a local directory and quickly get an idea of what synths I'm using the most.

    Side note: You may want to note that Firefox is not supported due to use of window.showDirectoryPicker

  • 0
    drbourbondrbourbon ItalyBeta Tester Posts: 55

    @Haila said:
    Thanks for the kind words! I'll keep tapping away with the goal of getting sample stats and broken sample detection going. Need to spend some time figuring out how to detect and parse the various firmware formats first :smile:

    I'll update the post here when I feel like something newsworthy has happened.

    My Deluge Commander is coded in java script and has some song, sound and kit parsing for sample identification. This supports 2.x, 3.x and 4.x formats. It's on github so feel free to grab the code you need.

  • 1
    HailaHaila FinlandPosts: 16

    @tomtobblestop Good point. I gave the front page a bit of love to make the browser support more obvious. Hopefully, the other browser will follow suit soon to make this a non-issue, alto the API was officially released only about a month ago.

    @drbourbon Thanks for the permission! I'm taking the painful route of strong typing the Deluge formats to TS so I think the majority of parsing will be my personal ditch to shovel. It took a month, but I rewrote everything to support separate parsing modes for different firmware versions. I'll definitely keep referring to your stuff for ideas, tho!

    The above rewrite is now mostly done so I think I'll soon get to sample usage stats and missing sample detection. Had to build a file browser for navigation as it was getting silly for multiple thousands of samples in a single long list 😅

  • 3
    HailaHaila FinlandPosts: 16
    edited June 2022

    Progress update #1 - Samples!

    Finally getting to one of the features I started building this thing for: meaningful visualization of sample usage to help trim the sample library. It's a bit raw, doesn't support audio tracks, and in general somewhat untested for edge cases, but...

    Delugr now tracks how samples are used and visualizes that in the new folder-based navigation. Folders with only un-used samples are also marked un-used to make it easier to figure out what folders can be culled away.

    For example, the samples in most of my "artists" folders are not used yet.

    And, just in case you don't remember what Kody Nielson's sample named "5" sounds like, I added a button to play the raw samples right in the browser. Spoiler: it's a low-fi mechanical click. I think.

    I'm also slowly figuring out how to meaningfully show the chain of a sample being used in various contexts. Coversely, the synth view can now also show direct links to individual samples to make navigation between things as easy as possible.

    All this was rather boring work so I procrastinated a bit by redoing the landing page. It now should have much more helpful error messages, feedback during folder analysis and sweet, sweet floating memory cards in the background 😅

    Next up: I'll probably take a stab at renaming and deleting files. Feedback welcome if you happen to find this useful!

    Post edited by Haila on
  • 0
    jensgjensg Berlin, GermanyBeta Tester Posts: 297

    tried this on mac it but got this message:

  • 0
    jensgjensg Berlin, GermanyBeta Tester Posts: 297

    after removing all of the junk on my sd card it throws this error:

  • 0
    jensgjensg Berlin, GermanyBeta Tester Posts: 297

    after removing the "offending" file, this happens. I guess you need to foolproof this way more. at least for my messy kind of card / content...

  • 0
    HailaHaila FinlandPosts: 16

    Thanks for reporting!

    The first one seems obvious for a fix, but the two latter ones need a bit of research. Any chance you could send me the offending files and I’ll have a look? teemu.haila@gmail.com is probably most convenient.

  • 1
    HailaHaila FinlandPosts: 16

    I just pushed out a few fixes to hopefully help with the above:

    • All dot files are now ignored completely.
    • echoinInput is now optional instead of a mandatory attribute (the other attributes will likely fail instead, I'm not sure how that part of the files works yet).
    • Files that fail parsing get added to a neat list instead of blocking the whole app. The error was probably more useful for me than you anyway 🤨

    About the missing 'osc1' in a sound... That's super unexpected. All my files always have osc1 defined. Seems like there's some kind of default fallback for an important property being completely undefined, but I'm not sure what that is. I guess I should try deleting properties like osc1 off of a sound and see what happens on the device.

    It would make sense that there's some firmware-specific default patch or something that "fills in the gaps" for undefined things, but I'm not sure if there's a clever way to verify and export that theory. Another rainy day puzzle?

  • 0
    Wolf_TWolf_T SpainPosts: 39

    On my pc it does not show songs done with firmwareVersion="4.0.0".

  • 0
    djAziddjAzid AmsterdamPosts: 199

    Well, it does say: "Firmware version 4.0.0 is not supported for kits."
    And it's still not supporting aiff files.
    The overview also needs a scrollbar because now you can't see everything below a certain amount.
    Some of the note data goes out of the grey area, like: noteData="0x00000000000000186414000000180000001864140000003000000018641400000048000000186414000000600000001864140000007800000018641400000090000000186414000000A8000000186414000000C0000000186414000000D8000000186414000000F00000001864140000010800000018641400000120000000186414000001380000001864140000015000000018641400000168000000186414"

    or

    preview="000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000F60900000000F60900FE0100FE0100000000000000FF0000000000F60900FE0100FC0003000000FF000000000000FF06B0004F000000000000F60009000000000000000000A5005A230214000000000000000000000000F600098000801C021C1C021C00FF06B0004F005AA50000000000000000000500FA005AA5005AA5000000000000005AA50000000500FA0000000009F600000000000000FF06B0004F00609F00000000000000609F60009F00609F00FC0300609F00000000609F60009F00609F00609F00609F00609F60009F00FF06B0004F00609F00609F00B04F00609F00609F00609F926D0000B04F00609F00609F00609F00609F00609F00609F926D0000609FFF0000B0004F00609F00000000000000609F00000000609F00000000609F00000000609F00000000609F00609F00609F00609F000000FF0000005AA5"

    No idea why the deluge has parameters that big anyway ¯_(ツ)_/¯

    Sample scanning takes forever, hope that can be optimized.

  • 0
    rezareza los angelesModerator, Beta Tester Posts: 603

    for the 'preview' value, each sequence of ~5 numbers represents one pad on the grid, so it's a one-to-one way of representing what colors are lit on the grid!

  • 0
    HailaHaila FinlandPosts: 16

    Finally got around to tinkering with this again. Thanks for the feedback!

    Some recent updates:

    • Firmware v4 files should load ok. Probably there are edge cases that I didn't find yet 🤨
    • AIFF files should work now for samples.
    • The raw data fields wrap instead of overflowing.
    • Synths now default to an empty patch and only then load data from a save file. That should handle the "parameter missing" errors. Not sure if this is how the device actually works, but it's good enough for this project's purposes atm.

    I've got some work done towards actually managing the sample files but it's not good enough yet to include in the public builds. I'll get to it a bit later!

  • 0
    norisoundsnorisounds swedenBeta Tester Posts: 28

    Just stumpled upon this and wanted to give a huge thanks!

  • 0
    Wolf_TWolf_T SpainPosts: 39

    I wanted to give it a try, this is now in the browser:
    "Parsing the folder contents...
    9397 files scanned"
    But nothing happens after that.Google Chrome on iMac Version 111.0.5563.110 (Official Build) (x86_64)

  • 0
    HailaHaila FinlandPosts: 16

    Thanks, @norisounds! Glad to hear someone else has a use for this project ✌️

    @Wolf_T Ouch, not great. I just now looked at the underlying code, and getting stuck in parsing without an error message is surprising. Maybe some infinite loop in parsing all the folders? It doesn't seem to happen with my memory card. Then again, I don't have the OLED screen and firmware, so maybe something new kills the code. Hard to know.

    I think the only way I can help is if you could send me a copy of your memory card so I can try to replicate the issue. Alternatively, you could try to (make a copy of the card and) delete some of your folders one at a time to see what folder and/or file might be causing the issue. A bit of a stab in the dark, I know 🤷‍♂️

  • 0
    HailaHaila FinlandPosts: 16

    @Wolf_T by luck, I happened to find another computer that took an unexpectedly long time to parse the files. I rewrote how the data is stored in memory, and at least on this one problematic machine, it now seems to not get stuck. Please try again if your issue got solved.

    I also put in some UI polish work, and now there are more icons all over the place and more play buttons next to samples. I think it improves legibility and reduces clicks when, for example, viewing synths that use multi-samples.

  • 0
    Wolf_TWolf_T SpainPosts: 39

    Problem found, it was due to some files in the synth folder. I deleted them, now it works.

  • 0
    HailaHaila FinlandPosts: 16

    I just pushed out support for drag & drop in the initial folder selection and a nicer sample preview with auto-play to make it a bit easier to browse a trillion samples quickly. Small steps!

  • 0
    HailaHaila FinlandPosts: 16
    edited June 2023

    Another small update: skipped files now show a button to use the new (and still experimental) browser API for file deletions. Easy way to clean up some junk files.

    This was the easiest place to learn how to use that particular API. I aim to implement this same thing for samples & sample folders for easier culling of un-used samples.


    Post edited by Haila on
  • 0
    idealflawidealflaw austinPosts: 7

    @Haila dumb question - do i have to pull the SD card out of the deluge and put it in my PC? or can i just USB up the deluge?

  • 0
    HailaHaila FinlandPosts: 16

    @idealflaw said:
    @Haila dumb question - do i have to pull the SD card out of the deluge and put it in my PC? or can i just USB up the deluge?

    Yep, get your memory card into a computer so the browser can access it.

    Deluge does not know how to talk to computers over USB.

  • 0
    BennyOgreBennyOgre The Woods near SeattlePosts: 2
    edited December 2023

    *EDIT: Welp, fifth time's a charm! Beautiful UI.

    Heyo.

    Using Chrome on MacOS, the site freezes when attempting to access the Samples section. I get a "Page Unresponsive" popup each time.

    Post edited by BennyOgre on
Sign In or Register to comment.