astrobase.checkplot.pkl_postproc module

This contains utility functions that support the checkplot pickle post-processing functionality.

astrobase.checkplot.pkl_postproc.update_checkplot_objectinfo(cpf, fast_mode=False, findercmap='gray_r', finderconvolve=None, deredden_object=True, custom_bandpasses=None, gaia_submit_timeout=10.0, gaia_submit_tries=3, gaia_max_timeout=180.0, gaia_mirror=None, complete_query_later=True, lclistpkl=None, nbrradiusarcsec=60.0, maxnumneighbors=5, plotdpi=100, findercachedir='~/.astrobase/stamp-cache', verbose=True)[source]

This updates a checkplot objectinfo dict.

Useful in cases where a previous round of GAIA/finderchart/external catalog acquisition failed. This will preserve the following keys in the checkplot if they exist:

comments
varinfo
objectinfo.objecttags
Parameters:
  • cpf (str) – The path to the checkplot pickle to update.
  • fast_mode (bool or float) – This runs the external catalog operations in a “fast” mode, with short timeouts and not trying to hit external catalogs that take a long time to respond. See the docstring for astrobase.checkplot.pkl_utils._pkl_finder_objectinfo() for details on how this works. If this is True, will run in “fast” mode with default timeouts (5 seconds in most cases). If this is a float, will run in “fast” mode with the provided timeout value in seconds.
  • findercmap (str or matplotlib.cm.ColorMap object) – The Colormap object to use for the finder chart image.
  • finderconvolve (astropy.convolution.Kernel object or None) – If not None, the Kernel object to use for convolving the finder image.
  • deredden_objects (bool) – If this is True, will use the 2MASS DUST service to get extinction coefficients in various bands, and then try to deredden the magnitudes and colors of the object already present in the checkplot’s objectinfo dict.
  • custom_bandpasses (dict) – This is a dict used to provide custom bandpass definitions for any magnitude measurements in the objectinfo dict that are not automatically recognized by the varclass.starfeatures.color_features function. See its docstring for details on the required format.
  • gaia_submit_timeout (float) – Sets the timeout in seconds to use when submitting a request to look up the object’s information to the GAIA service. Note that if fast_mode is set, this is ignored.
  • gaia_submit_tries (int) – Sets the maximum number of times the GAIA services will be contacted to obtain this object’s information. If fast_mode is set, this is ignored, and the services will be contacted only once (meaning that a failure to respond will be silently ignored and no GAIA data will be added to the checkplot’s objectinfo dict).
  • gaia_max_timeout (float) – Sets the timeout in seconds to use when waiting for the GAIA service to respond to our request for the object’s information. Note that if fast_mode is set, this is ignored.
  • gaia_mirror (str) – This sets the GAIA mirror to use. This is a key in the astrobase.services.gaia.GAIA_URLS dict which defines the URLs to hit for each mirror.
  • complete_query_later (bool) – If this is True, saves the state of GAIA queries that are not yet complete when gaia_max_timeout is reached while waiting for the GAIA service to respond to our request. A later call for GAIA info on the same object will attempt to pick up the results from the existing query if it’s completed. If fast_mode is True, this is ignored.
  • lclistpkl (dict or str) – If this is provided, must be a dict resulting from reading a catalog produced by the lcproc.catalogs.make_lclist function or a str path pointing to the pickle file produced by that function. This catalog is used to find neighbors of the current object in the current light curve collection. Looking at neighbors of the object within the radius specified by nbrradiusarcsec is useful for light curves produced by instruments that have a large pixel scale, so are susceptible to blending of variability and potential confusion of neighbor variability with that of the actual object being looked at. If this is None, no neighbor lookups will be performed.
  • nbrradiusarcsec (float) – The radius in arcseconds to use for a search conducted around the coordinates of this object to look for any potential confusion and blending of variability amplitude caused by their proximity.
  • maxnumneighbors (int) – The maximum number of neighbors that will have their light curves and magnitudes noted in this checkplot as potential blends with the target object.
  • plotdpi (int) – The resolution in DPI of the plots to generate in this function (e.g. the finder chart, etc.)
  • findercachedir (str) – The path to the astrobase cache directory for finder chart downloads from the NASA SkyView service.
  • verbose (bool) – If True, will indicate progress and warn about potential problems.
Returns:

Path to the updated checkplot pickle file.
Return type:

str
astrobase.checkplot.pkl_postproc.finalize_checkplot(cpx, outdir, all_lclistpkl, objfits=None)[source]

This is used to prevent any further changes to the checkplot.

TODO: finish this.

Use this function after all variable classification, period-finding, and object xmatches are done. This function will add a ‘final’ key to the checkplot, which will contain:

  • a phased LC plot with the period and epoch set after review using the times, mags, errs after any appropriate filtering and sigclip was done in the checkplotserver UI
  • The unphased LC using the times, mags, errs after any appropriate filtering and sigclip was done in the checkplotserver UI
  • the same plots for any LC collection neighbors
  • the survey cutout for the object if objfits is provided and checks out
  • a redone neighbor search using GAIA and all light curves in the collection even if they don’t have at least 1000 observations.

These items will be shown in a special ‘Final’ tab in the checkplotserver webapp (this should be run in readonly mode as well). The final tab will also contain downloadable links for the checkplot pickle in pkl and PNG format, as well as the final times, mags, errs as a gzipped CSV with a header containing all of this info.

Parameters:
  • cpx (dict or str) – This is the path to the checkplot dict or pickle file to process.
  • outdir (str) – This is the directory to where the final pickle will be written. If this is set to the same dir as cpx and cpx is a pickle, the function will return a failure. This is meant to keep the in-process checkplots separate from the finalized versions.
  • all_lclistpkl (str or dict) – This is the path to the pickle or the dict created by astrobase.lcproc.catalogs.make_lclist() with no restrictions on the number of observations (so ALL light curves in the collection). This is used to make sure all neighbors of this object in the light curve collection have had their proximity to this object noted.
  • objfits (str or None) – If this is not None, should be a file path to a FITS file containing a WCS header and this object from the instrument that was used to observe it. This will be used to make a stamp cutout of the object using the actual image it was detected on. This will be a useful comparison to the usual DSS POSS-RED2 image used by the checkplots.
Returns:

The path to the updated checkplot pickle file with a ‘final’ key added to it , as described above.
Return type:

str
astrobase.checkplot.pkl_postproc.parallel_finalize_cplist(cplist, outdir, objfits=None)[source]

This is a parallel driver for finalize_checkplot, operating on a list of checkplots.

Parameters:
  • cplist (list of str) – This is a list of paths of all checkplot pickles to process and run through the finalization process.
  • outdir (str) – The directory to where the finalized checkplot pickles will be written.
  • objfits (str) – Path to the FITS file containing a WCS header and detections of all objects observed by the actual instrument that obtained the light curves. This should generally be a high quality ‘reference’ frame so that all of the objects whose checkplots we’re finalizing (in cplist) can be seen on the frame.
Returns:

Dict indicating the success/failure (as True/False) of the checkplot finalize operations for each checkplot pickle provided in cplist.
Return type:

dict
astrobase.checkplot.pkl_postproc.parallel_finalize_cpdir(cpdir, outdir, cpfileglob='checkplot-*.pkl*', objfits=None)[source]

This is a parallel driver for finalize_checkplot, operating on a directory of checkplots.

Parameters:
  • cpdir (str) – This is the path to the directory containing all the checkplot pickles to process and run through the finalization process.
  • outdir (str) – The directory to where the finalized checkplot pickles will be written.
  • objfits (str) – Path to the FITS file containing a WCS header and detections of all objects observed by the actual instrument that obtained the light curves. This should generally be a high quality ‘reference’ frame so that all of the objects whose checkplots we’re finalizing (in cplist) can be seen on the frame.
Returns:

Dict indicating the success/failure (as True/False) of the checkplot finalize operations for each checkplot pickle provided in cplist.
Return type:

dict