Processing Ajax...

Title
Close Dialog

Message

Confirm
Close Dialog

Confirm
Close Dialog

Confirm
Close Dialog

Rick Fox's profile on WallpaperFusion.com
I'm just curious really, but that was sparked by often seeing a string of images, 'randomly' chosen, that I know are all in the same folder.

That might make you think, "so what?", but I have perhaps 60GB of images in a huge multiple folder structure, and most of the folders are of similar size. So, to see several of them one after another from the same folder, and to notice this quite often, seems not so random. Perhaps it's just a fluke though.

It's certainly not an issue if it isn't really random, but I might change the way my folders are organised if I know how files are chosen, to 'assist' the randomness and make sure all my files get an even chance.

PS. DF is the best thing since I first got multiple monitors in NT4 workstation!
Dec 7, 2014  • #1
User Image
tryingout
72 discussion posts
I would love to get an answer to this question too. I asked it many times in an earlier thread (DF v6) that focused in part on "randomness" problems, now largely fixed. But as helpful as Keith was on solving problems, he never answered this question.

I think it must be a secret, or maybe I have to start a bonfire and do a "code dance" to the "code gods" that live in the ethereal internet cloud.... ;-)
Dec 9, 2014  • #2
Keith Lammers (BFS)'s profile on WallpaperFusion.com
It builds one giant list of all images in all of the sub-folders, but it also keeps a history of which images have been shown, so that they won’t be shown again until all other images have been shown. In some cases that can have weird results, like if you add images faster than they can be shown, you’ll always be seeing the newer images and never any old ones. If you want to try clearing the image history, just delete the %localappdata%\WallpaperHistoryV4.db file and see if that helps :)

Please let me know if you still have any trouble at all!
Dec 9, 2014  • #3
User Image
tryingout
72 discussion posts
Thanks for the effort but this doesn't really answer the question of how "random" is determined. For example:

- Does DF simply display images in 1 to N order from the big list, checking first to be sure it is not in the "recently displayed" history? (This seems very code inefficient.)

- Does DF choose a random number between 1 and N from a random number generator and check if the image is in the recent history list, and keep doing this until it finds one not recent?

- To make the code a bit more efficient, maybe DF makes a random selection 1 to N, and then works only in that folder until a non-recent image is found? If something like this is the case, are subsequent look-ups done randomly or alphabetically within the folder until a non-recent image is found?

- What if all images in the big list are also on the recent history list, what does DF do?

- How does the "Find Local Random Images using Less Memory" affect this?

I hope these examples make the question a bit clearer?

Thx.
Dec 9, 2014 (modified Dec 9, 2014)  • #4
Jon Tackabury (BFS)'s profile on WallpaperFusion.com
DisplayFusion builds a big list of images from all folders that aren't in the history, then randomly picks one and adds it to the image history. Next time through, the same image can't get picked again. If there are no images to choose from, probably because they're all in the history, DisplayFusion resets the history to nothing and starts the process all over again. In this case, it would be possible to see the same image before and after the history reset, but this is very rare.
Dec 10, 2014  • #5
Rick Fox's profile on WallpaperFusion.com
Thanks for the replies.

Hmm... so, if DF simply builds one list, without regard to the folders that images are in, then things should appear random; psedo-random to be a pedant.

Seeing a string of images from the same folder, like I keep doing, must be just randomness showing that it can appear non-random even when it is. Assuming it all works as expected of course.

Seeing imges repeated is unlikely to be an issue to me, with perhaps 100,000 images to go at. Of course, I'm now going to be watching for that, lol.

I do not seem to have a WallpaperHistoryV4.db file in the appdata\local folder, or anywhere else for that matter. I did a full search. I'm actually glad because I try to minimise writes to my system HD (512GB SSD). The first thing I thought when I read that statement was, "can I move it?"

I even have DF redirected to a ramdisk for building the wallpaper.

I'm curious what the "Find Local Random Images using Less Memory" does too. At the moment, FD is using about 70MB of RAM in total. Hah - Firestorm is using 250MB just to view these pages. The switch is probably not of use to me, I'm just curious as I saw it in the settings the other day.
Dec 10, 2014  • #6
Rick Fox's profile on WallpaperFusion.com
Jon, your reply came in while I was typing my previous post.

It sounds like you are saying that DF rebuilds the 'select from' list every time a wallpaper change is due. Would this be a lot of work if I have so many images? (at least 100,000 and growing, 350,000 if I run the whole lot)

I do sometimes have a momentary system slowdown as wallpapers are changed, and the mouse cursor flickers to 'busy'. It doesn't always happen; I'm trying to get it to right now, changing every ten seconds. It does happen often enough that it's definitely linked to the wallpaper change, to the point that I'll sometimes think "Wallpaper change happening" just before it actually changes.

It likely depends on what else I'm doing, some of my applications can be quite intensive especially when I have a couple of VMs running.

My CPU graph is showing core 0 giving a good, wide peak to about 50% every wallpaper change. No other applications loaded of course.

My CPU is an i7 at 3.4GHz and I have 16GB RAM and an MSI GTX970 graphics card, so I shouldn't be too short on resources.

Just wondering if rebuilding such a large list might not be a good idea in my case. I've always put the momentaary glitch down more to Windows itself, and it isn't bad enough to be a real problem (so I've never even thought to mention it before) at least with my usual set of 100,000.

I'm going to use my full set and see if it makes a difference...
Dec 10, 2014  • #7
Jon Tackabury (BFS)'s profile on WallpaperFusion.com
It does unfortunately build the entire list each time it does a change. On smaller collections this is fine, but on larger collections it can use too much memory. We added an advanced setting for people with large image collections, you can read more about it here:
http://www.displayfusion.com/AdvancedSettings/#wallpaper-lowmemorylocalrandom

It can be enabled in the DisplayFusion Advanced Settings window, in the Wallpaper section. Hopefully this helps you out. :)
Dec 10, 2014  • #8
Rick Fox's profile on WallpaperFusion.com
Ah-ha - so that setting is of interest to me.

I'll give it a go, although I can't get my PC to do the glitch at the moment, even running on the 360,000 set. Mustn't be pushing it hard enough.

Thanks!
Dec 10, 2014  • #9
Rick Fox's profile on WallpaperFusion.com
I've just noticed a setting that might be of even more use in my case, "Don't Store Image History". I'm hardly likely to get the same image cropping up noticably often, so I'll give this setting a go too, or instead of the "use low memory" one.

If it stops DF having to rebuild the list every change, and just picks at random from the full list, that;s fine with me. Edit to add: I've just recalled that I couldn't find the mentioned history file anyway.

I did notice, just before turning on the "low memory" setting, that DF had shown a couple of images from the same, small subfolder branch under the 'root' of the images. After turning on the setting, I got another four or five from the same branch, but different subfolders in that branch. That branch probably only has about 200 images in it and they are very specific, so easy to spot. There's definitely a pattern there, unless randomness is being very un-random.

...it's gone to that branch again as I'm typing, after moving elsewhere for five minutes. Two images so far from that branch...three...four...five...and it's moved on again elsewhere.

I'll carry on playing around.
Dec 10, 2014 (modified Dec 10, 2014)  • #10
Rick Fox's profile on WallpaperFusion.com
OK,. ten minutes later and DF has gone back to that same branch three (now four as I'm typing) times, showing strings of images just from there up to six images.

After that, it always goes to another particualr branch under the root (this one has helicopters in it) and shows one image from there, and there is a definite pattern to the following images/folders too; I'm pretty sure they are coming from the same places... yep, the very same image has now shown four times from another folder... and back to that original branch with a string of images from it.

I would say about 20 % of the pictures I'm seeing are from the same branch of a couple of hundred, out of 350,000.

I'm also definitely seeing many (same) pictures shown many times, the one I mentioned above at four is now five. (now six! Seven. Eight) and others are similar..

If this is the effect of that "low memory" setting (I guess it is) it's really not very good. I'm going to turn it off now and just leave the "don't save history" on.

On the positive, the CPU peaks are now only 25-30%. But random it isn't.

------------------

Summary

Low Memory mode definitely has a lower CPU impact, the peaks top out at 25%-30%. Normal Mode gives much fatter and higher peaks at 50% ish. Low Memory mode is certainly not random though, nothing like it, at least with my folder arrangement. I guess if it were a single folder it would be, since it seems folder-centric, but that's just not practical with so many pictures.

I've not seen any difference yet with the "Don;t save history" setting.
Dec 11, 2014 (modified Dec 11, 2014)  • #11
User Image
tryingout
72 discussion posts
Jon wrote:

"DisplayFusion builds a big list of images from all folders that aren't in the history, then randomly picks one and adds it to the image history."

After writing my comment yesterday I thought of this option too. In my case I have the images in 3 parent directories, one for each monitor, with time interval changes of 15, 20, and 15 seconds. I've "only" got something over 50,000 images though, not 350,000. But even with my smaller library, that's still enumerating 20,000 + 20,000 every 15 seconds, and something over 10,000 every 20 seconds. Then converting every filename to an index code as stored in the WallpaperHistoryV4 file, doing a massive compare/delete until a net undisplayed file list remains, and selecting one from this subset to display.

Wow. That's a lot of CPU cycles!

If Rick's observations are correct though, and I think they are because they agree with mine, this final selection from the subset is not quite as random as "random" would suggest.

I'd like to offer a suggestion for your next release (v7.1?): Do a directory enumeration every say 60 minutes (or better, make it user configurable in the advanced settings). Do the huge comparison/delete process and come up with a net not-displayed list, and just work off this smaller (little or a lot depending on how old/full the history is) until the next cycle. Then as each image is displayed all the code has to do is update the WallpaperHistoryV4 file and randomly select the next image from the smaller list. You don't even need to keep the non-displayed-image list in memory, just add a new table to WallpaperHistoryV4 and simply delete each image entry as it is shown and move it to the already-shown history table.

And, if you do the initial writes to the not-shown list in random order (not sure what this would cost CPU-wise though), then at each wallpaper change it would be just an increment to the next entry instead of code doing a random selection every time. And you don't have to write the entire list, just enough entries to last until the next cycle (60 min or whatever).

And (yes, another one ;-) this can be done asynchronously in the background to minimize CPU impact.

Even with a smaller image library this will save huge CPU time. And you can probably get rid of the use-less-memory option too.

Just some thoughts.

And I agree with Rick that DF rocks. And with the improvements you've already made I think it's one of the best multi-monitor wallpaper programs out there.
Dec 11, 2014 (modified Dec 11, 2014)  • #12
Rick Fox's profile on WallpaperFusion.com
Some good points there, tryingout.

Having once earned a crust as an analyst/programmer, I can well imagine the various issues around trying to do a one-size-fits-all solution to this. There really isn't one.

For me, simply not bothering with the history, keeping just one big list built at startup of all files, and choosing randomly from that regardless of the underlying directory structure would be perfect. I suppose the fileset would need checking periodically (preferably user defined) to see if anything has changed, but even that could be optional or done just on profile change for instance. That's probably about as simple as it can get.

Perhaps, since I also use some smaller sets on my primary monitor, the ability to set whether the history method is user per-profile would be nice. If not, I would still rather see a few duplicates than have to use the history method across the board.

Oddly, having turned off the history, I've now found that WallpaperHistoryV4.db file, under the displayfusion folder in appdata\local. Perhaps I (and my system search, with copy/pasted filename) somehow missed it. Occam's razor would suggest so.

Last modified time on that file is last night at 23:58, about the time I turned history off so that seems to be working. Still, the big CPU spikes would suggest that all the list building is still going on. I'm not running the "Low Memory" setting any more, it was doing my head in.
Dec 11, 2014 (modified Dec 11, 2014)  • #13
Jon Tackabury (BFS)'s profile on WallpaperFusion.com
You're right about the "use less memory" setting, it isn't nearly as "random" as the normal mode. We do plan to implement caching in 7.1, we've had a few customers with large image collections run into performance issues. We ran out of time to make this big of a change for 7.0, but don't worry, we plan to fix it. :)
Dec 11, 2014  • #14
Rick Fox's profile on WallpaperFusion.com
That sounds good to me. To be honest, I don't see much of a performance issue most of the time. When I do, I'm pushing my PC quite hard anyway and probably shouldn't be running something as trivial as a wallpaper change in the background, lol.

It's always good to have things improved though, so I'll look forward to the changes.
Dec 11, 2014  • #15
Rick Fox's profile on WallpaperFusion.com
Just a follow up on this. I've had history disabled since my last post, and not using the low memory option.

While I can't prove anything, I swear that I'm seeing more randomness in what pictures get shown. I'm seeing ones from some big folders (10,000's images) that I've hardly ever seen before, but should have given the size of the folders. Maybe it's just a placebo effect, but it seems real.

That would indicate that the history-keeping method wasn't working randomly and, as I always sort of suspected, that it was influenced by the folder structure. I could be wrong though and the god of randomness might just be having a laugh at my expense.

The brief lag when the wallpaper is changing is still definitely going on, with the busy cursor flickering on momentarily and the occasional obvious slowdown in graphics and processing intensive applications. It's most obvious when I'm panning around a complex 3D animated scene with a six-axis controller, and the momentary lag causes my viewpoint to jerk suddenly.
Dec 15, 2014  • #16
Keith Lammers (BFS)'s profile on WallpaperFusion.com
@Rick, for the busy cursor, can you try enabling the Wallpaper > "Use Internal Wallpaper Processing" option in the DF Settings > Advanced Settings window?
Dec 15, 2014  • #17
User Image
tryingout
72 discussion posts
I want to reinforce @Rick's comment re the effect of turning off wallpaper history. After reading his comment I turned-off the history and saw an immediate improvement in the so-called random selection. On 3 monitors over 15-20 minutes, all the images were ones I'd never seen before for almost the first 10 minutes, and 3 or 4 over the next ~10 minutes. Quite an improvement.

I had a long discussion in another thread http://www.displayfusion.com/Discussions/View/screen-saver-freezing-windows-7-here-too/?ID=48427163-d544-4b15-be90-a0190ccfcaa6#13 with Keith that started on a different topic and shifted to wallpaper selection. They did make quite a few improvements that made the "random" part a lot better, but I eventually stopped pushing the issue because I thought I was the only one with this problem. While it's true that the squeaky wheel gets the grease, if it squeaks too much it gets thrown away! And I didn't want to get labeled as a crackpot/troll with some esoteric problem no one else cared about.

In that thread, Keith mentioned when a "folder" gets enumerated. He may have been speaking generally, but I asked about this in that thread several times and never got a satisfactory answer. That's why I appreciated @Jon Tackabury's remark above clarifying this. But like @Rick, I'm not sure I completely believe it; something is still off re random and wallpaper history.

What I find surprising is that I'm still getting CPU spikes at every wallpaper change. Without the directory enumeration and database comparisons, I would have thought these would go away. Building a new background/wallpaper image and loading it shouldn't be making these kinds of demands on a modern quad-core system. While directory enumeration of 20k+ (or 100k+) files will take time, it's nothing compared to doing a database comparison process.

@Jon Tackabury did say above that they were planning to look into caching for DF v7.1. That will certainly help, but I still think the random issue and/or wallpaper history needs to be looked at too.

As I've been typing this I've also been watching the wallpapers, and I definitely see a big improvement in the randomness with no history.
Dec 15, 2014  • #18
Rick Fox's profile on WallpaperFusion.com
@Keith, I'll give that a try for a while and see how it goes. I can immediately see a small drop in CPU spikes. I guess the exact load depends on what DF has to do to each individual picture, but I watched it for long enough in Resource Monitor to see a definite trend as shown in the graphs below. CPU 0 shows the difference.

@tryingout, it is good to have someone making similar observations, and yours appear to be closely parallel to mine. The difference in 'randomness' really is obvious when you watch for a while, isn't it?

Dec 16, 2014  • #19
Jon Tackabury (BFS)'s profile on WallpaperFusion.com
The caching coming in 7.1 will help with all of this. You'll be able to use the history to keep the randomness, without the performance penalty of using it. And you'll be able to have large collections without having to poll for images every time. :)
Dec 17, 2014  • #20
User Image
tryingout
72 discussion posts
@Jon,

Looking forward to this. It'll be the bee's knees!
Dec 18, 2014  • #21
User Image
tryingout
72 discussion posts
Any update on when this will be released? DF 7.2 beta 2 has been out for a while now. Thx.
Feb 22, 2015  • #22
Jon Tackabury (BFS)'s profile on WallpaperFusion.com
7.1 and 7.2 B2 should have this fix already. Are you still experiencing performance issues?
Feb 23, 2015  • #23
User Image
tryingout
72 discussion posts
I haven't noticed any performance differences re CPU usage. As for randomness, no improvement, it is the same as before. I still think you have a bug somewhere in your code. Either directories are not being enumerated completely every time, or the pool from which images are being selected is getting constrained by something other than history. Maybe a buffer is getting stomped on sometimes. It's not unusual for 4 images from the same directory (out of a dozen+) that differ only by a suffix term (e.g., pic1, pic2, pic3, pic4, pic5, pic6) to show up in a row. Statistically the odds of that happening once are remote, but multiple times in the 60 days since I last posted on this thread, well... maybe I should be buying lottery tickets! ;-)

Random thought:
Here's an odd thought: are the filenames at any time getting converted to the old DOS 8.3 format, selected, then converted back to the "long filename"? Long shot I know, but if that is happening and not done properly, that would lead to problems. Windows still has a fair amount of old code in it, and one of the earliest lessons every new programmer learns is, if it ain't broke, don't fix it!

I did try turning off the history function per @Rick Fox's comments, and while things were better for a while, after a few days I re-enabled history because repeated images were fewer over the longer term.

You've explained what you intend and believe the code is doing, but it's not. Or maybe it is sometimes but not others. Hopefully @Rick Fox can chime in here and relate his experiences. He has a larger image library than I do, 350K versus my 75K, and uses a faster cycling rate.
Feb 23, 2015  • #24
Jon Tackabury (BFS)'s profile on WallpaperFusion.com
That is very strange, I have a few questions for you so we can try to solve this:
- Do you have the "Show in alphabetical order" setting enabled in the wallpaper window?
- Do you have any settings modified in the "Wallpaper" section in the "Advanced Settings"?

We aren't using any functions that would use the 8.3 filenames, so I don't think that is the issue. Thanks!
Mar 2, 2015  • #25
User Image
Chad Biggerstaff1
1 discussion post
Thought I'd chime in as I have a huge collection of wallpapers and have not noticed an issue with radomness so far. If I understood a past poster he had a dozen or so folders with images, myself I have 1,594 folders with 151,942 files. Perhaps a lack of randomness is caused by a lack of folders? Any chance the random code actually hits a directory randomly before looking at files?
I'm running v7.2 with history on, I've changed the history to go for 180 days, and have it set to not change the wallpaper randomly when accessing the wallpaper dialog.
Jul 28, 2015  • #26
Subscribe to this discussion topic using RSS
Was this helpful?  Login to Vote(-)  Login to Vote(-)