AllSkEye: Sky cam/meteor detection

[Al.Grant]

Here's a quick First Look at a bit of software I've been using for the last couple of days whilst setting up an all sky cam to look for meteors.

I've only been looking at it for a couple of days but already like what I've seen.

 

The software is   AllSkyEye https://www.allskeye.com/

 

AllSkEye designed for running all sky cameras so it has more 'all sky' features than software that is designed more for general cctv monitoring.
There is a feature list here https://www.allskeye.com/features/ and the ones I've tried so far all seem to work pretty well.

It has native support for ZWO , Altair and Starlight Express cameras (also ASCOM support) and it recognised my old model ZWO ASI120mm straight away.  No messing around with drivers.

 

It's primarily designed for full time 24/7 use, however, in the short time I've been looking at it I've had no problem with an ad hoc setup.
Some users of older ASI120's have reported some issues with various things, but I haven't seen any of these and I'm currently using it attached to a 12m USB2 active hub plugged into a fairly low spec laptop running Win10.

 

It doesn't record video directly.  

First it records a series of .fit 'Acquisition' files, these are converted to either .jpg or .png and then these are converted to video. This can be all automated or the video can be created manually.  
Happily the dev has included the option to automatically delete old files once they have been converted, which is good considering the first 'indoor' test run I did recorded over 29,000 .fit files!
These were then converted to .jpg ..... so quite a few files and I soon turned ON the auto delete option.   

There is a reduce image size option which keeps converted files sizes quite low, so things aren't as bad as it initially sounds.  With all this conversion going on I ran a small windows resource monitor and was pleased to see it didn't get anywhere near stressing out the laptop. 

 

One thing I have noticed (and it makes perfect sense) is that the acquisition of .fits is far faster than the conversion process. So you can see the acquisition folder fill up quickly whilst the conversion folder can be quite a way behind.  If you manually stop the system that's where it will stay.  There is a 'Processing Only' option that will run the conversion process without taking more .fits images and I've found this to be very useful.   Only needed if you manually start/stop the run, if done via timers then it sorts itself out.

 

 

The are a number of timing options for capturing images.  
It can run constantly 24/7 or fixed times set by the user or it can run from ## number of minutes after sunset until ## number of minutes before sunrise and the options I've tried seem to work well.

 

There is an option for meteor detection which looks interesting.  Instead of motion detection it adopts a 'line detection' method.  
If it sees a streak across the sky then it will mark it as a possible meteor and take a separate capture with the option to mark it with a circle.  So aircraft or trees blowing around are less likely to trigger a detection.   
I had it out for a few hours the other night (not all night as I need a better enclosure) and  when my son turned his room light on it caused a things near the window to light up and some of these  showed up as a 'line' and it marked it.  I did watch a few aircraft cross the screen and it ignored these.  So I know that feature works but I'm not interested in when his light comes on or off! 

There is an option I've not yet tried yet (one of many) that allows you to set a capture area, efffectively ignoring areas of the field of view and perhaps not capturing bedroom lights coming on!

 

It's easy to import your local weather (from openweathermap) and display this in a tab in the program, you can also overlay this on the image along with other text like lat/long or things like logo images or a compass rose.

 

Captured for posterity as it shows 'clear sky'

If you have a website then it has built in ftp functionality (I've not tested) which can upload a video when completed or you can tell it how often you want the 'latest image' uploading
The AllSkEYE website has a live image here - I think it updates every 60sec:
https://www.allskeye.com/allskeye-live/

 

I've tried these and got them working, although I'm seeing an issue with the weather overlay on reduced image size files.  Speaking to the Dev it looks like a bug and he's already working on it.

 

There are options to map bad/hot/cold pixels but I've not tried that and there is even an option to overlay a star map (not plate solving) but even the dev says it takes some fiddling to get right.

Some of the other options may take a bit more digging to get it working ..... if I actually need them! 

 

Overall, for a first look I'm quite impressed and it's been easy to get up and running with just a fairly basic set up and I think I'll be sticking with AllSyEye rather than iSpy
 

I still need to get my head around the auto-exposure settings, but I've almost got it sorted.

 

This is a quick .jpg from the test I did last night.  I was mostly looking to sort out the camera focus - clouds got in the way - and check it was OK on the long USB cable.

 

Anybody already running or thinking of running an all sky camera might want to give this a look.

 

 

Al

 

 

[Al.Grant]

 

Hi Graham,

On the right side of the Run System tab, if you select ASI ZWO does it find your camera ok?

Ensure the default directories have been created.  Settings tab > right hand side 'Create Default Folders'

 

As long as it's seeing the camera there isn't too much to check to at least get you started.

There is a copy of my base settings here:  https://drive.google.com/open?id=1XVcayAHiM4fWeqHwZu8VTrGe57S1DvKe

 

I've not yet got the auto exposure sorted properly - somethings not quite right with those settings but it's a start point!

 

This is set to Fixed times and AllSkEye will record from 20:00 until 05:00, create the latest image every 60secs.

This can be changed to 'Night' or Continuous.

 

You'll need to change things like your home location 'Image Acquisition' Observation Long & lat and also my name is in the FITS header.

 

The easiest way is to type observer into the Settings search bar and it will filter on the correct sections.

 

To make adjustments to the image you may want to search Gain and also Exposure.

 

The 'Loop' option at the bottom of the Run System tab is usful for generating a preview image whilst tweaking these settings without actually saving any image files.

 

I find this helpful when trying to work out what setting does what  https://www.allskeye.com/userdoc/index.html

 

 

Mine is being given a quick test with the new dew heater and dome tonight.  Only a temporary position as we've got scaffolding all over the house so it will need to move tomorros.

 

 

Al

 

 

[+Graham]

Brilliant Al thanks.

It sees the Camera and that works ok.

I have set up the folders ok.

Just needed the rest of what to do.

Thanks

[+Graham]

Well I have it running.

I have put a limit of 1000mb on all the folders.

It was filling up too quick 😅

 

have not got a latest image despite running it for a few mins.

 

How did you get your info to display on the image -- time , date , lat , long ect

 

Thanks

[Al.Grant]

Quote

have not got a latest image despite running it for a few mins.

Have a look in the Latest Image Settings section and tick the 'Continuous' box.

Otherwise I think it should do it when actually recording - can't check as mine is running now and you can't go into the settings whilst it's running.

 

Quote

How did you get your info to display on the image -- time , date , lat , long ect

In the settings search box type Overlay and it will bring up a number of places options where you'll see (Collection) - Click that and there are options to add a text overlay.

There are a few of these and they can be a bit of a pain to find at first.

 

There are some placeholders that go in the displayed text box.  If you go to https://www.allskeye.com/userdoc/index.html and search on Overlay it gives examples of the placeholders that go there.

 

It's things like    {ObservationDateTime}   {ExposureTime}   {SiteLatitude}   {SiteLongitude}

 

I'm not sure why mine is showing 10ms exposure when it's set to 10sec exposure .... I've probably got something set wrong.

 

I'm pretty certain I got one ove before everything clouded over

 

Definately not an aircraft, I've had quite a few of those and they look totally different.

 

[+Graham]

Thats awesome 👍

Thanks Al I will give that a look.

[Al.Grant]

Here's an example of what I have in the Image Conversion text overlay - - I forgot I have them in a text file so I can copy/paste the same info in different places.

 

Lat: {SiteLatitude}{cr}Long: {SiteLongitude}{cr}Time: {ObservationDateTime}{cr}Exp.: {ExposureTime}ms

It also explains why mine is showing ms

 

There is also a section that says Displayed Text Startpoint X & Y -  0 & 0 is the top left corner of the screen. 

You need to enter the number of pixels to offset it to place it where you want on the screen ..... takes a bit of trial and error.

The weather info on mine was placed for a reduced image size, so I need to move it over.

 

AllSkEye's config takes a bit of getting used to, but so far I like how it works.

Still things I want to adjust like the Detection parameters and also there is an option to take reference dark frames and bad pixel maps which I want to look at next.

 

 

Edited December 17, 2019 by Al.Grant

[+Graham]

I cannot find any way to control the size of the Acquisition folder.

It has only been running for a few mins and it is up to 2.4 g already 

 

Reset the exposure time from 10 ms up to 5 seconds and reduced the image size.

Much better 

 

Thats interesting.

Now I have changed the exposure time the fit image auto deletes out of the acquisition folder so there is no build up of data in it 

 

I have the weather updates running now.

How did you get it to display on the image.

Did you have to write a new script 

 

I have stuck the camera out of the bathroom window and set it running.

Still cloudy down here but at least it will test the software out.

Edited December 18, 2019 by tuckstar
Merged posts.

[Al.Grant]

Hi

 

8 hours ago, Graham said:

Reset the exposure time from 10 ms up to 5 seconds and reduced the image size.

Much better 

 

Thats interesting.

Now I have changed the exposure time the fit image auto deletes out of the acquisition folder so there is no build up of data in it 

I thought I had the same but found the FIT images were auto deleting even though I thought they weren't.

 

It's just that it takes a few seconds to convert a FIT to a .jpg or .png and with an exposure of only a few ms the FIT's in was obviously far greater than the .jpg out so there was a backlog (quite a large one!).   With longer exposures the conversion process has a better chance of keeping up with the incoming FITs images.

 

If you find that you get a backlog once the collection has stopped, go to the Run System tab and in the bottom right ther is a 'Processing Only' button.

This will process any outstanding FIT images to clear the backlog without capturing an new ones.

 

7 hours ago, Graham said:

I have the weather updates running now.

How did you get it to display on the image.

Did you have to write a new script 

Nice that you have the weather info updating.

To get it to display you just add another text overlay.

In the settings go to    Image Overlay > Text Overlays > (Collection)

It the popup box add another text overlay (pity these can't be renamed) and enter more of those placeholders.

Mine currently shows:

 

Weather: {Weather}{cr} Cloud: {CloudCover}{cr} Wind Speed {WindSpeed}{cr} Wind Direction: {WindDirectionCardinal}{cr} Temp: {Temperature}{cr} Humidity: {Humidity}{cr} Dew Point: {DewPoint}

 

You'll need to move the x/y position of the overlay to ensure it doesn't get over laid over an exisiting one.

 

If you enter a placeholder for info that isn't available for your area then it will display an N/A.  I just used ones that I knew were showing up on the Weather tab.

 

NOTE:   There is a bug!

I found a bug with the weather overlay when using the reduced image size option.   The same settings work correctly on a full size image.

The overlay will display but all options will show N/A. 

I posted this on the support forum and the dev says it must be a bug and is looking at it.

 

Mine ran as expected last night, I was particularly happy that the new dew heater kept the dome clear all night long.

It automatially processed the video this morining and I've also manually generated a video as I wanted to check what fps setting works best.

I'm going to cut the video down as once the cloud rolls in it stays there for the rest of the night - I'll put it up on Youtube once it's done.

 

Next on the list is the dark reference image and the hot pixel map to see if I can get a better image.   

The dark image is automatically subtracted when the conversion takes place, so I suspect this might slightly increase the conversion time .... but we'll see.

 

 

Hope your's worked last night.

 

Al

 

[+Graham]

Morning al.

Thanks for the code.

Mine ran brilliantly last night even though I had literally just stuck it out of the top of the bathroom window and tied it in place.

Just got to tweak the focus and I will be in business.

Gave up with the auto exposure.

Sorted it by opening up the ASI camera tab and reducing the gain down to 10 to compensate for the long exposure.

Here is the video it produced for me when it stopped running at 6 am this morning.

 

 

[Al.Grant]

Nice one Graham .... certainly not bad for out of the bathroom window 👍

 

Mine was set to run from 20:00 - 05:00 but I generated another video only showing roughly the first hour, after that the rest of the night was just cloud.

At about 8-9 secs there is a brief flash of a meteor on the right hand side, that's the one in the image I posted earlier.

This is an old camera and there are definitely lots of hot pixels, so I'm going to try a pixel map later.

 

There are some things I want change in the config and overlays, but so far from the few tests I've done the software works pretty good,  even if the config does take a bit of getting used to.

 

This video has 10sec exposures and the ASI120 gain was set at 20

 

 

 

Now I know it's working I've backup up the settings so if I mess it up with any future tweaks I can easily go back.

 

Al

 

 

 

Edited December 18, 2019 by Al.Grant

[+Graham]

That is awesome Al.

And I thought I had a lot of air traffic to contend with 😅

 

I have to admit this software is brilliant.

Well done you for finding it.

 

Now about to venture outside and fix the camera to the house.

Going to be pointing it west so it should get a fair amount of sky in North to South.

 

How do you back up the settings.

Edited December 18, 2019 by Graham

[Al.Grant]

Cheers Graham,

 

Being fairly close to East Midlands airport the air traffic here is busier than I'd like but it does depend on which direction they are landing/taking off from East Mids.

 

Now I've got a heater working and the fogging issue sorted I'll be building a better enclosure and looking for a better spot on the house to mount it - however it will always be mounted in a way that

I can quickly and easily take it down as I fully intend to take it with me when we go over to Snowdonia (usually 3 days per month) where I'll get to run it under some very dark skies

and far, far fewer aircraft 😄    That said, I'm already considering a 2nd camera to I can have one fixed at home and one mobile.

 

I've had a quick go following the instructions to create the Hot/cold pixel map but it's not picking any up .... and I know there are lots on this camera!

 

 

[+Graham]

 

Well I have mine all set up and ready for tonight.

Climbing around on ladders is probably not really advisable for me with my Vertigo but it had to be done. 😅

Heater is all wired up.

It turns on automatically when the temp drops.

I have put it out side the window where my computer is so I can reach it to make final adjustments on the focus.

 

End result.

 

 

[Al.Grant]

Looks good!

 

The forecast here is cloud & rain, so I've not got my hopes up of getting anything tonight.  I will run it though as it at least tests the software and settings.

 

Pity really as I've now got the bad pixel map working and configured and wanted to see what sort of difference it makes.

I've also made some changes to the Detection settings so it should give less false readings.

It detects lines on some of the houses at the edge of the image, there is a feature to mask out areas but as it's only in a temporary position I'm not going to look at that.

 

 

[Al.Grant]

Graham,

 

Just noticed your comment about backing up the settings ....

You may have found it by now, but if not then go to the Settings tab and on the right side there is an option to Export or Import.

 

With Export you just select a folder and it generates an xml file with all your setting in. 

 

 

[+Graham]

Brilliant stuff thanks Al

 

Cannot get the weather stuff to work.

There must be a setting I am missing in the weather tab.

Edited December 18, 2019 by Graham

[Al.Grant]

Graham,

Have you got a weather api from https://openweathermap.org/ 

Once you have an api then you add it in the Weather settings tab.  That should bring in the weather for your location.

 

 

 

[Al.Grant]

 

Currently my all sky camera is running from a laptop that sits in the conservatory (well hidden) and I log into it remotely from my main laptop in the house using Anydesk.

I had a spare stick pc at work not doing much so I've decided to try running the cam and AllSkEye from this.

 

Having installed the ZWO drivers and AllSkEye I then copied over my current settings.  It found the camera straight away and seems to be working.

It's running headless, so no monitor, keyboard or mouse, just power and the camera usb cable.

 

 

It's been up and running now for an hour capturing FIT's files every 0.3ms and converting them without any issues, obviously there is a backlog of FITs files to convert but that happens

on my normal laptops.

 

Processor wise there doesn't seem to be an issue (not yet tried video conversion) but these things don't have massive amounts of storage space.

At the moment it's just running on the internal 64gb, with Win10 and a few other bits installed I currently have 44gb free - I think there should be enough for an overnight run but if

things get tight I can add another 64 or 128 gb micro SD drive.

 

Stick PC:

Intel Atom x5-Z8350 Quad Core processor 

4GB RAM

64Gb internal drive.  Expandable via micro ssd

Built in wifi

1x USB2  &  1x UBS3 port

It has Intel graphics and is supposedly capable of 4k video playback.

 

 

 

 

[+Graham]

Looks good Al.

How do you download or even look at what is on it.

 

[Al.Grant]

I use Anydesk (free for personal use) to remote log in and it brings up the Stick PC desktop in a window on my main laptop.

I also use Anydesk to do the same with my Spectrum Lab laptop.  It also allows me to log in from work to check on them.

 

Anydesk is pretty much the same as Teamviewer but I find it runs a bit quicker.

Functionality is pretty much the same.  Remote login and desktop usage and also ftp files between laptops.

 

I only found Anydesk becuase Teamviewer blocked my account saying I was using it commercially - I wasn't! 

What I actually did was log in from work to check my home laptops.  This is listed on their website in their acceptable terms.

By the time they agreed and unblocked my account I had found Anydesk and never gone back to Teamviewer

 

 

[Al.Grant]

 

Whilst trying out this Stick PC I decided to take a look at the file acquisition v conversion.

 

I'm currently using .png as the converted format and these are much larger than the .jpg

Conversions checked over a 1 min and 5 min time period work out at a constant 30 conversions per min, so one every 2 secs.

Once a FIT file is converted it gets deleted but with my current settings they're coming in at 62 per min.

 

After running for a couple of hours I have a backlog of 4211 Fits images each at 2404kb in size

With these rates I'm using about 5Gb storage space per hour.  

 

This shouldn't be anywhere near as bad when imaging at night as any exposure over 2 secs and the conversion will be able to keep up.

Recently I've been using 10 sec exposures and no FIT backlog, so that space will be saved. I could save even more space by using .jpg as the conversion format.

 

I've just done a video conversion of the last 2 hours of rain  and although it the conversion was quite slow the Stick PC did complete the video it without any complaints or getting hot.

 

I'm going to give it a timed run for a few hours tonight and see how it goes.

 

 

[+Graham]

Al

I have just tried the Detection bit and it raises the conversion time considerably.

In fact I would say it scanned one image every minuet or so.

This meant an ever growing backlog of captured images even when running at 10 seconds.

Once switched off the captures were back to being processed in a couple of seconds so no backlog.

I always capture in Jpeg .

 

This is the first image in tonight.

Think I have the focus pretty much nailed now.

 

 

Edited December 19, 2019 by Graham

[Al.Grant]

Hi Graham,

 

You're right about the line detection, I didn't have that enabled as I'm not happy with the default settings and need to look closer at it.

I see in the settings it says it is processor intensive and it can take 10-20 secs per image.  You're clearly seeing it take longer and I've no reason to think mine would be less. 

You can create a mask so the detection only runs on a certain area, this supposedly helps it just look at the sky without looking at the low horizon where there might be more light pollution .....  mine certainly picks up lines on houses & trees.  I might give this a try and see what it's like with or without the masking.

 

----------------

 

I ran my camera last night using the Stick PC and it worked without any problem.

I ran it without line detection and using .jpg for the images and generated a video at the end of the run

It ran from 21:30 until 05:00.  With all the FITs images processed but keeping all the jpg's and the generated video file I still have 45Gb free ... I uninstalled a few bits last night to give me more free space.

 

Running from the small Stick PC when we're away will give me a few more options for siting the camera.

I can put it outside in a waterproof housing with power running to it and then log in from the laptop.

 

 

 

 

[+Graham]

Sounds promising