hats.io.paths

hats.io.paths#

Methods for creating partitioned data paths

Attributes#

`PARTITION_ORDER`
`PARTITION_DIR`
`PARTITION_PIXEL`
`MARGIN_ORDER`
`MARGIN_DIR`
`MARGIN_PIXEL`
`HIVE_COLUMNS`
`DATASET_DIR`
`PARTITION_INFO_FILENAME`
`PARQUET_METADATA_FILENAME`
`PARQUET_COMMON_METADATA_FILENAME`
`DATA_THUMBNAIL_FILENAME`
`POINT_MAP_FILENAME`
`SKYMAP_FILENAME`

Functions#

`dataset_directory`(→ upath.UPath)	Create path pointer for the dataset directory. This will not create the directory.
`pixel_directory`(→ upath.UPath)	Create path pointer for a pixel directory. This will not create the directory.
`get_healpix_from_path`(...)	Find the pixel_order and pixel_number from a string like the
`dict_to_query_urlparams`(→ str)	Converts a dictionary to a url query parameter string
`pixel_catalog_file`(→ upath.UPath)	Create path pointer for a pixel catalog file. This will not create the directory
`get_partition_info_pointer`(→ upath.UPath)	Get file pointer to `partition_info.csv` metadata file
`get_common_metadata_pointer`(→ upath.UPath)	Get file pointer to _common_metadata parquet metadata file
`get_parquet_metadata_pointer`(→ upath.UPath)	Get file pointer to _metadata parquet metadata file
`get_data_thumbnail_pointer`(→ upath.UPath)	Get file pointer to data_thumbnail parquet file
`get_point_map_file_pointer`(→ upath.UPath)	Get file pointer to point_map.fits FITS image file.
`get_skymap_file_pointer`(→ upath.UPath)	Get file pointer to skymap.fits or skymap.K.fits FITS image file.

Module Contents#

PARTITION_ORDER = 'Norder'[source]#

PARTITION_DIR = 'Dir'[source]#

PARTITION_PIXEL = 'Npix'[source]#

MARGIN_ORDER = 'margin_Norder'[source]#

MARGIN_DIR = 'margin_Dir'[source]#

MARGIN_PIXEL = 'margin_Npix'[source]#

HIVE_COLUMNS[source]#

DATASET_DIR = 'dataset'[source]#

PARTITION_INFO_FILENAME = 'partition_info.csv'[source]#

PARQUET_METADATA_FILENAME = '_metadata'[source]#

PARQUET_COMMON_METADATA_FILENAME = '_common_metadata'[source]#

DATA_THUMBNAIL_FILENAME = 'data_thumbnail.parquet'[source]#

POINT_MAP_FILENAME = 'point_map.fits'[source]#

SKYMAP_FILENAME = 'skymap.fits'[source]#

dataset_directory(catalog_base_dir: str | pathlib.Path | upath.UPath | None) → upath.UPath[source]#

Create path pointer for the dataset directory. This will not create the directory.

The directory name will take the HiPS standard form of:

<catalog_base_dir>/dataset/

Parameters:

catalog_base_dirstr | Path | UPath | None: base directory of the catalog (includes catalog name)

Returns:

UPath: dataset directory name

Create path pointer for a pixel directory. This will not create the directory.

One of pixel_number or directory_number is required. The directory name will take the HiPS standard form of:

<catalog_base_dir>/dataset/Norder=<pixel_order>/Dir=<directory number>

Where the directory number is calculated using integer division as:

(pixel_number/10000)*10000

Parameters:

catalog_base_dirstr | Path | UPath | None: base directory of the catalog (includes catalog name)
pixel_orderint: the healpix order of the pixel
pixel_numberint | None: the number of the healpix pixel at pixel_order
directory_numberint | None: directory number (or inferred from pixel number)

Returns:

UPath: directory name

get_healpix_from_path(path: str) → hats.pixel_math.healpix_pixel.HealpixPixel[source]#

Find the pixel_order and pixel_number from a string like the following:

Norder=<pixel_order>/Dir=<directory number>/Npix=<pixel_number>.parquet

NB: This expects the format generated by the pixel_catalog_file method

Parameters:

pathstr: path to parse

Returns:

HealpixPixel: Constructed HealpixPixel object representing the pixel in the path. INVALID_PIXEL if the path doesn’t match the expected pattern for any reason.

dict_to_query_urlparams(query_params: dict | None = None) → str[source]#

Converts a dictionary to a url query parameter string

Parameters:

query_params: dict | None: (Default value = None) set of URL query parameters.

Returns:

str: query parameter string to append to a url

pixel_catalog_file(catalog_base_dir: str | pathlib.Path | upath.UPath | None, pixel: hats.pixel_math.healpix_pixel.HealpixPixel, query_params: dict | None = None, npix_suffix: str = '.parquet') → upath.UPath[source]#

Create path pointer for a pixel catalog file. This will not create the directory or file.

The catalog file name will take the HiPS standard form of:

<catalog_base_dir>/Norder=<pixel_order>/Dir=<directory number>/Npix=<pixel_number>.parquet

Where the directory number is calculated using integer division as:

(pixel_number/10000)*10000

Parameters:

catalog_base_dirstr | Path | UPath | None

base directory of the catalog (includes catalog name)

pixelHealpixPixel

the healpix pixel to create path to

query_params: dict | None

(Default value = None) Params to append to URL. Ex:

{'cols': ['ra', 'dec'], 'fltrs': ['r>=10', 'g<18']}

npix_suffix: str

(Default value = “.parquet”) extension for the parquet file (or / if a directory)

Returns:

UPath: catalog file name

get_partition_info_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to partition_info.csv metadata file

Parameters:

catalog_base_dir: str | Path | UPath: base directory of the catalog (includes catalog name)

Returns:

UPath: File Pointer to the catalog’s partition_info.csv file

get_common_metadata_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to _common_metadata parquet metadata file

Parameters:

catalog_base_dir: str | Path | UPath: base directory of the catalog (includes catalog name)

Returns:

UPath: File Pointer to the catalog’s _common_metadata file

get_parquet_metadata_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to _metadata parquet metadata file

Parameters:

catalog_base_dir: str | Path | UPath: base directory of the catalog (includes catalog name)

Returns:

UPath: File Pointer to the catalog’s _metadata file

get_data_thumbnail_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to data_thumbnail parquet file

Parameters:

catalog_base_dir: str | Path | UPath: base directory of the catalog (includes catalog name)

Returns:

UPath: File Pointer to the catalog’s data_thumbnail file

get_point_map_file_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to point_map.fits FITS image file.

Parameters:

catalog_base_dir: str | Path | UPath: base directory of the catalog (includes catalog name)

Returns:

UPath: File Pointer to the catalog’s point_map.fits FITS image file.

get_skymap_file_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath, order: int | None = None) → upath.UPath[source]#

Get file pointer to skymap.fits or skymap.K.fits FITS image file.

Parameters:

catalog_base_dir: str | Path | UPath: base directory of the catalog (includes catalog name)
order: int | None: (Default value = None) desired order for the map, if looking for a down-sampled map.

Returns:

UPath: File Pointer to the FITS image file.