NormalizeScaleGradient Script

[GordonH]

John, 

Thanks, but the same error just appeared.

Gordon

[Spaced Out Bill]

On 10/15/2022 at 12:52 AM, John Murphy said:

I am on holiday for a week. I will have a look when I get back.

Regards, John

Hello John,

I hope you had a great time on holiday!

Do you have any idea what is causing the blotchyness in the NSG image?

Thank you!

[John Murphy]

8 hours ago, Spaced Out Bill said:

Hello John,

I hope you had a great time on holiday!

Do you have any idea what is causing the blotchyness in the NSG image?

Thank you!

I will be back on Sunday.

• The most likely cause is that the uneven background existed on the reference frame. Did you use NSG's Blink dialog to select the most suitable reference frame?
• The gradient dialog can be used to blink between the reference image and the corrected target. What you see is what you get. There are STF controls to add contrast to make the result easier to see. The smoothness controls should be modified until you have the correction you desire.

Beyond this, I would require all your registered images, otherwise it is not possible to pinpoint precisely what your situation is.

Regards, John Murphy

Edited October 19, 2022 by John Murphy

[John Murphy]

The NormalizeScaleGradient website address has been simplified to:

https://www.normalizescalegradient.net/

 

Although you can still also access it via the original web address:

https://sites.google.com/view/normalizescalegradient

 

Regards, John Murphy

 

[rockenrock]

Hi John,

In my PI v1.8.9 on PC, the NSG script version is showing up as 2.2.0, not 2.2.3 or 2.2.3. I have nsg.astropills.it in the repository.

So kindly advise what is needed to get updated. I have not purchased the NSGxml.

 

Roger

[rockenrock]

On 11/1/2022 at 10:24 AM, rockenrock said:

Hi John,

In my PI v1.8.9 on PC, the NSG script version is showing up as 2.2.0, not 2.2.3 or 2.2.3. I have nsg.astropills.it in the repository.

So kindly advise what is needed to get updated. I have not purchased the NSGxml.

 

Roger

Sorry John,

I had failed to update Pixinsight to 1.8.9-1 because I was travelling overseas, then I forgot it.

Today,after downloading the PI 1.8.9-1 installation package it updates all the repositories. Then restart PI and all the updated items are installed and available.

So now I have 2.2.3! 

Cheers,

     Roger

[John Murphy]

A reminder of the NSG installation process:

 

Installing NSG and NSGXnml
(1) Upgrade to the latest version of PixInsight (currently PixInsight 1.8.9-1): https://pixinsight.com/dist/
(2) Add the NSG repository to PixInsight:
RESOURCES > Updates > Manage Repositories
Press 'Add' button and enter the repository URL: https://nsg.astropills.it

(Note that the repository URL does not contain www)
(3) Remember to restart PixInsight so that the updates are installed!
(4) If you use a Mac, you will need to install NSGXnml manually. The NSG website explains how to do this.
(It is installed automatically for both Windows and Linux.)

Activate NSGXnml License
To activate the license, run the NSGXnml process:
PROCESS > Image Calibration > NSGXnml
Enter the license email and key, then press 'Activate'.

NSG Website:
https://www.normalizescalegradient.net/

 

Regards, John Murphy

[macbates]

The latest version of WBPP (2.5.4) introduced the Autocrop function, located in the Lights tab. Since my normal use of WBPP with NSG is to disable/uncheck Local Normalization and Image Integration in the Lights tab, this also makes the Autocrop check box unavailable. How does this new WBPP “functionality” play with NSG? Is there a way to retain the autocrop functionality while still using all of NSG? I haven’t tried the new WBPP yet, but I’m going to be doing so soon and wanted to see if you had any advice for using NSG while retaining the autocrop functionality.

 

Thanks in advance for any advice,

 

 - Ken

[John Murphy]

21 hours ago, macbates said:

The latest version of WBPP (2.5.4) introduced the Autocrop function, located in the Lights tab. Since my normal use of WBPP with NSG is to disable/uncheck Local Normalization and Image Integration in the Lights tab, this also makes the Autocrop check box unavailable. How does this new WBPP “functionality” play with NSG? Is there a way to retain the autocrop functionality while still using all of NSG? I haven’t tried the new WBPP yet, but I’m going to be doing so soon and wanted to see if you had any advice for using NSG while retaining the autocrop functionality.

 

Thanks in advance for any advice,

 

 - Ken

I have just updated to the latest WBPP version, but I can't find the Autocrop function in the Lights tab! What does the new Autocrop function do?

Edited November 24, 2022 by John Murphy

[macbates]

I believe that it's supposed to automatically crop out any stacking artifacts. There was a brief description of the functionality in the announcement, and I think that it generates additional files to support this functionality. I'm attaching a screenshot that shows the checkbox to enable/disable autocrop. That said, I haven't tried it so I don't know how it works or whether it's worth doing.

[John Murphy]

Hi Ken,

I think that the main use case for Auto crop is to ensure that SpectrophotometricColorCalibration (SPCC) still works correctly if the image normalization was poor. With approximate normalization, you get 'step artifacts' at the edges of an image because not all images contribute to these areas. Any normalization faults then become apparent and cause these artifacts. This will then cause problems in the photometry.

 

However, if the image normalization is accurate (for example, you used NormalizeScaleGradient), then these step artifacts disappear. True, the regions where not all images contributed will have lower SNR, but the photometry will still produce the correct answer because the astronomical signal in these areas will still have the correct strength.

 

So if you are using NSG, you don't need Auto crop. I would keep the whole image, which then gives you more flexibility when you decide on your final crop. Often a small SNR drop in edge regions is not significant, and extending the area covered may be beneficial.

Regards, John Murphy

https://www.normalizescalegradient.net/

[macbates]

Thanks, John. That makes perfect sense. I use NSG on all my image processing, and will probably just use PCC instead of SPCC since the brief tests I've done show no noticeable differences to my eye.

[John Murphy]

After NSG and ImageIntegration, you can use PCC or SPCC. Both should work extremely well. Regards, John Murphy

https://www.normalizescalegradient.net/

[Sedat]

 

Hi John,

 

My drizzled images don't seem to be corrected by NSG. Is this a pilot error ? Thanks.

 

Sedat

 

 

[Minusman]

Hi John, a question about the NSG script. I had now observed in different data sets, when I look at the transmission diagram, that with different binnings the images with Bin2x2 are clearly superior in the weightings to the images with Bin1x1. Although exposure time, gain are the same and moon phase is similar. The Bin1x1 images are then rejected with the standard settings of the Image Rejection almost all in relation to Bin2x2 images. Is it then advisable to process the images with the same binning in the NSG script separately and then combine them with Pixelmath?

[John Murphy]

On 11/26/2022 at 4:08 PM, Minusman said:

Hi John, a question about the NSG script. I had now observed in different data sets, when I look at the transmission diagram, that with different binnings the images with Bin2x2 are clearly superior in the weightings to the images with Bin1x1. Although exposure time, gain are the same and moon phase is similar. The Bin1x1 images are then rejected with the standard settings of the Image Rejection almost all in relation to Bin2x2 images. Is it then advisable to process the images with the same binning in the NSG script separately and then combine them with Pixelmath?

One cause of this issue is the noise estimate which PixInsight stores in the FITS header. PixInsight calculates this noise estimate during pre-processing (before registration). However, if the image is resized after the noise estimate has been stored (due to software binning or registration with different pixel scales), the noise estimate is no longer correct. PixInsight should update the noise estimate depending on how much bigger or smaller the image became. I have previously recommended this change, but it was not taken up...

 

The other problem is that when PixInsight resizes an image, it does not conserve flux. For example, if you apply 2x binning, to conserve flux you would need to increase the pixel brightness by a factor 4. Instead it stays the same. The transmission diagram is based on star flux, so software binning, or registration with images of mixed pixel scales, will affect the displayed transmission value.

 

Yes, processing the images with the same binning in the NSG script separately and then combining them using PixelMath would be a reasonable solution.

 

Regards, John Murphy

[John Murphy]

On 11/26/2022 at 3:34 PM, Sedat said:

 

Hi John,

 

My drizzled images don't seem to be corrected by NSG. Is this a pilot error ? Thanks.

 

Sedat

 

 

NSG creates '.xnml' LocalNormalization data files. These files specify the scale and gradient corrections. Both ImageIntegration and DrizzleIntegration need these files.

 

When NSG invokes ImageIntegration, it automatically adds these '.xnml' files to it. Once ImageIntegration has finished, you need to setup the DrizzleIntegration process. This time you need to manually add the '.xnml' files by using the  'Add L.Norm. Files' button.

 

Regards, John Murphy

https://www.normalizescalegradient.net/

[Sedat]

Of course! I cannot believe I overlooked this step. Thank you John! The result was excellent after adding the normalization files NSG produced.

 

Cheers

 

Sedat

[John Murphy]

NormalizeScaleGradient 2.2.4 has been released 😀

A big thank you to Marco for hosting the NSG repository: https://nsg.astropills.it

 

I have also released my very first YouTube video: An introduction to NormalizeScaleGradient:

https://www.youtube.com/@NormalizeScaleGradient

I will be adding more very soon!

 

Release notes:

NSG 2.2.4 (Requires PixInsight 1.8.9-1 or later and NSGXnml 1.0.4)

• Updated user interface to make individual reset buttons more intuitive.
• Changed Help and Reset from tool buttons to buttons with icons.
• Added numeric field to show the number of unrejected samples.
• Added ability to display NWEIGHT points in the weight graph.
• If DATE-OBS entries are all equal, sort on full file path instead.
• Updated help to 2.2.4

Regards, John Murphy

https://www.normalizescalegradient.net/

https://www.youtube.com/@NormalizeScaleGradient

[Phillyo118]

Hi John,

 

Firstly thank you for the NSG script, I find it really useful and improves my images over the normal WBPP script even with the LN functionality included within WBPP.

 

However, for the last few weeks when I've tried to use NSG, Pixinsight has crashed midway through the script with the error "A breakpoint has been reached (Error 0x80000003)".

 

Any ideas what might be causing that? PI works in every other instance.

 

Thanks,

Phil.

[John Murphy]

Hi Phil,

Microsoft describes this error message here, and provides possible solutions you can try:

https://answers.microsoft.com/en-us/windows/forum/all/how-do-i-get-rid-of-the-error-exception-break/cb222a25-89c2-4b36-910e-9130bc686cba

 

It appears that it can be caused by a memory or file I/O error.

 

I would recommend checking the NSG option "Normalization data" and deselect "Normalized images" as this will reduce the amount of file I/O

 

Regards, John Murphy

https://www.normalizescalegradient.net/

https://www.youtube.com/@NormalizeScaleGradient

[Phillyo118]

Thank you for your response John. I'll take a look at the I/O error! Frustrating.

[zerolatitude]

Hi, just starting imaging again after months of rain.

 

Great updates - the videos are also excellent.

 

Thanks again.  

[zerolatitude]

Question re using NSG on lights with different filters.

 

OSC capture, some with a UHC, some with an LPro. The LPro with much wider bandpass is obviously brighter and has more gradients than the UHC.

 

I usually integrate these all at once without any LN. Now I'm wondering what it would be like if I were to use NSG.

 

Since the background is so different, even a reference frame that works for all would seem difficult to get. I was thinking NSG separately for each filter set and then integrate.

 

Does that sound right? I will of course try it out, but wondered if there's any conceptual direction?

 

Thanks

[John Murphy]

On 12/13/2022 at 9:29 AM, zerolatitude said:

Question re using NSG on lights with different filters.

 

OSC capture, some with a UHC, some with an LPro. The LPro with much wider bandpass is obviously brighter and has more gradients than the UHC.

 

I usually integrate these all at once without any LN. Now I'm wondering what it would be like if I were to use NSG.

 

Since the background is so different, even a reference frame that works for all would seem difficult to get. I was thinking NSG separately for each filter set and then integrate.

 

Does that sound right? I will of course try it out, but wondered if there's any conceptual direction?

 

Thanks

There are a few different situations:

RGB Filters

If we were to normalize RGB filters to the same image (for example, all normalized to a green image), they would be scaled so the stars have the same brightness in all images, and the color of galaxies or nebula would be seen as a gradient to be removed. This is why we never include more than one RGB filter in the target list!

 

Stacked luminance + Luminance from Stacked RGB image

You have used NSG and have produced a stacked luminance, stacked R, G, B and produced their resulting RGB image. You have extracted the luminance from the stacked RGB, and wish to combine this with the stacked luminance. Your images are still linear (unstretched). In this case, you can use NSG with just two images: the stacked luminance and the luminance extracted from the RGB image. NSG will correctly normalize the two images, and then invoke ImageIntegration which will combine the two images with correct weights (NWEIGHT). Never just combine two stacked images using PixelMath; this will not produce an optimum SNR in the result. Always use NSG instead.

 

Red and Ha Filter

This will not work well either. Before normalization, the nebula is likely to have the same brightness in both the Red and Ha filters. But the stars will be much fainter in the Ha image. The brightness scale calculated from the stars will be completely invalid for the nebula. The result is the application of an invalid scale factor, which is then compensated by an invalid gradient (offset) correction. This is why you never want to mix broadband filters with narrow band filters in NSG's target list.

 

Luminance and light pollution filters

In this case it depends on your target image. If you are imaging galaxies (provided they don't have strong Ha emissions), then it is OK to mix luminance with a light pollution filter, or different light pollution filters. Just choose the image with a small, and more importantly, smooth gradient as the reference. Ignore the red warning text within NSG's target list. Why does it work in this case? Well galaxies are made up of stars, so the effect of the different filters is approximately equivalent to a change in transmission.

 

BUT, if the image contains emission nebula, then you can't mix and match Luminance with light pollution filters, or mix different light pollution filters.

 

Hope this makes it clear, John Murphy

https://www.normalizescalegradient.net/

https://www.youtube.com/@NormalizeScaleGradient