The AstroStat Slog » wavelet

Wavelet-regularized image deconvolution

hlee — Fri, 12 Jun 2009 20:47:36 +0000

A Fast Thresholded Landweber Algorithm for Wavelet-Regularized Multidimensional Deconvolution
Vonesch and Unser (2008)
IEEE Trans. Image Proc. vol. 17(4), pp. 539-549

Quoting the authors, I also like to say that the recovery of the original image from the observed is an ill-posed problem. They traced the efforts of wavelet regularization in deconvolution back to a few relatively recent publications by astronomers. Therefore, I guess the topic and algorithm of this paper could drag some attentions from astronomers.

They explain the wavelet based reconstruction procedure in a simple term. The matrix-vector product w_x= Wx yields the coefficients of x in the wavelet basis, and W^TWx reconstructs the signal from these coefficients.

Their assumed model is

y=Hx_orig + b,

where y and x_{orig} are vectors containing uniform samples of the original and measured signals; b represents the measurement error. H is a square (block) circulant matrix that approximates the convolution with the PSF. Then, the problem of deconvolution is to find an estimate that maximizes the cost function

J(x) = J_data(x)+ λ J_reg(x)

They described that “this functional can also interpreted as a (negative) log-likelihood in a Bayesian statistical framework, and deconvolution can then be seen as a maximum a posteriori (MAP) estimation problem.” Also the description of the cost function is applicable to the frequently appearing topic in regression or classification problems such as ridge regression, quantile regression, LASSO, LAR, model/variable selection, state space models from time series and spatial statistics, etc.

The observed image is the d-dimensional covolution of an origianl image (the characteristic function of the object of interest) with the impulse response (or PSF). of the imaging system.

The notion of regularization or penalizing the likelihood seems not well received among astronomers based on my observation that often times the chi-square minimization (the simple least square method) without penalty is suggested and used in astronomical data analysis. Since image analysis with wavelets popular in astronomy, the fast algorithm for wavelet regularized variational deconvolution introduced in this paper could bring faster results to astronomers and could offer better insights of the underlying physical processes by separating noise and background more in a model according fashion, not simple background subtraction.

[ArXiv] Sparse Poisson Intensity Reconstruction Algorithms

hlee — Thu, 07 May 2009 16:14:39 +0000

One of [ArXiv] papers from yesterday whose title might drag lots of attentions from astronomers. Furthermore, it’s a short paper.
[arxiv:math.CO:0905.0483] by Harmany, Marcia, and Willet.

Estimating f under “Sparse Poisson Intensity” condition is an frequently appearing topic in high energy astrophysics data analysis. Some might like to check references in the paper, which offer solutions to compressed sensing problems with different kinds of sparsity, minimization approaches, and constraints on f.

Apart from the technical details, the first two sentences from the conclusion,

We have developed computational approaches for signal reconstruction from photon-limited measurements – a situation prevalent in many practical settings. Our method optimizes a regularized Poisson likelihood under nonnegativity constraints

tempt me to study and try their algorithm.

[ArXiv] 4th week, May 2008

hlee — Sun, 01 Jun 2008 03:59:15 +0000

Eight astro-ph papers and two statistics paper are listed this week. One statistics paper discusses detecting filaments and the other talks about maximum likelihood estimation of satellite images (clouds).

[astro-ph:0805.3532] Balan and Lahav
ExoFit: Orbital Parameters of Extra-solar Planets from Radial Velocities (MCMC)
[astro-ph:0805.3983] R. G. Carlberg et al.
Clustering of supernova Ia host galaxies (Jackknife method is utilized).
[astro-ph:0805.4005] Kurek, Hrycyna, & Szydlowski
From model dynamics to oscillating dark energy parametrisation (Bayes factor)
[astro-ph:0805.4136] C. Genovese et al.
Inference for the Dark Energy Equation of State Using Type Ia Supernova Data
[math.ST:0805.4141] C. Genovese et al.
On the path density of a gradient field (detecting filaments via kernel density estimation, KDE)
[astro-ph:0805.4342] C. Espaillat et al.
Wavelet Analysis of AGN X-Ray Time Series: A QPO in 3C 273?
[astro-ph:0805.4414] Tegmark and Zaldarriaga
The Fast Fourier Transform Telescope
[astro-ph:0805.4417] A. Georgakakis et al.
A new method for determining the sensitivity of X-ray imaging observations and the X-ray number counts
[stat.AP:0805.4598] E. Anderes et al.
Maximum Likelihood Estimation of Cloud Height from Multi-Angle Satellite Imagery

Mexican Hat [EotW]

vlk — Wed, 28 May 2008 17:00:38 +0000

The most widely used tool for detecting sources in X-ray images, especially Chandra data, is the wavelet-based wavdetect, which uses the Mexican Hat (MH) wavelet. Now, the MH is not a very popular choice among wavelet aficianados because it does not form an orthonormal basis set (i.e., scale information is not well separated), and does not have compact support (i.e., the function extends to inifinity). So why is it used here?

The short answer is, it has a convenient background subtractor built in, is analytically comprehensible, and uses concepts very familiar to astronomers. The last bit can be seen by appealing to Gaussian smoothing. Astronomers are (or were) used to smoothing images with Gaussians, and in a manner of speaking, all astronomical images already come presmoothed by PSFs (point spread functions) that are nominally approximated by Gaussians. Now, if an image were smoothed by another Gaussian of a slightly larger width, the difference between the two smoothed images should highlight those features which are prominent at the spatial scale of the larger Gaussian. This is the basic rationale behind a wavelet.

So, in the following, G(x,y;σ_x,σ_y,x_o,y_o) is a 2D Gaussian written in such that the scaling of the widths and the transposition of the function is made obvious. It is defined over the real plane x,y ε R² and for widths σ_x,σ_y. The Mexican Hat wavelet MH(x,y;σ_x,σ_y,x_o,y_o) is generated as the difference between the two Gaussians of different widths, which essentially boils down to taking partial derivatives of G(σ_x,σ_y) wrt the widths. To be sure, these must really be thought of as operators where the functions are correlated with a data image, so the derivaties must be carried out inside an integral, but I am skipping all that for the sake of clarity. Also note, the MH is sometimes derived as the second derivative of G(x,y), the spatial derivatives that is.

The integral of the MH over R² results in the positive bump and the negative annulus canceling each other out, so there is no unambiguous way to set its normalization. And finally, the Fourier Transform shows which spatial scales (k_x,y are wavenumbers) are enhanced or filtered during a correlation.

[ArXiv] 5th week, Apr. 2008

hlee — Mon, 05 May 2008 07:08:42 +0000

Since I learned Hubble’s tuning fork^[1] for the first time, I wanted to do classification (semi-supervised learning seems more suitable) galaxies based on their features (colors and spectra), instead of labor intensive human eye classification. Ironically, at that time I didn’t know there is a field of computer science called machine learning nor statistics which do such studies. Upon switching to statistics with a hope of understanding statistical packages implemented in IRAF and IDL, and learning better the contents of Numerical Recipes and Bevington’s book, the ignorance was not the enemy, but the accessibility of data was.

I’m glad to see this week presented a paper that I had dreamed of many years ago in addition to other interesting papers. Nowadays, I’m more and more realizing that astronomical machine learning is not simple as what we see from machine learning and statistical computation literature, which typically adopted data sets from the data repository whose characteristics are well known over the many years (for example, the famous iris data; there are toy data sets and mock catalogs, no shortage of data sets of public characteristics). As the long list of authors indicates, machine learning on astronomical massive data sets are never meant to be a little girl’s dream. With a bit of my sentiment, I offer the list of this week:

[astro-ph:0804.4068] S. Pires et al.
FASTLens (FAst STatistics for weak Lensing) : Fast method for Weak Lensing Statistics and map making
[astro-ph:0804.4142] M.Kowalski et al.
Improved Cosmological Constraints from New, Old and Combined Supernova Datasets
[astro-ph:0804.4219] M. Bazarghan and R. Gupta
Automated Classification of Sloan Digital Sky Survey (SDSS) Stellar Spectra using Artificial Neural Networks
[gr-qc:0804.4144]E. L. Robinson, J. D. Romano, A. Vecchio
Search for a stochastic gravitational-wave signal in the second round of the Mock LISA Data challenges
[astro-ph:0804.4483]C. Lintott et al.
Galaxy Zoo : Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey
[astro-ph:0804.4692] M. J. Martinez Gonzalez et al.
PCA detection and denoising of Zeeman signatures in stellar polarised spectra
[astro-ph:0805.0101] J. Ireland et al.
Multiresolution analysis of active region magnetic structure and its correlation with the Mt. Wilson classification and flaring activity

A relevant post related machine learning on galaxy morphology from the slog is found at svm and galaxy morphological classification

< Added: 3rd week May 2008>[astro-ph:0805.2612] S. P. Bamford et al.
Galaxy Zoo: the independence of morphology and colour

Wikipedia link: Hubble sequence

[ArXiv] 1st week, Apr. 2008

hlee — Sun, 06 Apr 2008 15:10:15 +0000

I’m very curious how astronomers began to use Monte Carlo Markov Chain instead of Markov chain Monte Carlo. The more it becomes popular, the more frequently Monte Carlo Markov Chain appears. Anyway, this week, I added non astrostatistical papers in the list: a tutorial, big bang, and biblical theology.

[astro-ph:0803.4089] R. Trotta
Bayes in the sky: Bayesian inference and model selection in cosmology (Bayesian cosmology tutorial).

[astro-ph:0804.0070] W. Cui et al.
An ideal mass assignment scheme for measuring the Power Spectrum with FFTs

[astro-ph:0804.0155] L. Wang et al.
Timeline analysis and wavelet multiscale analysis of the AKARI All-Sky Survey at 90 micron

[astro-ph:0804.0278]L. Colombo and E. Pierpaoli
Model independent approaches to reionization in the analysis of upcoming CMB data

[astro-ph:0804.0285]L. Vergani et al.
Dark Matter – Dark Energy coupling biasing parameter estimates from CMB data

[astro-ph:0804.0294] A. Romeo et al.
Discreteness Effects in Lambda Cold Dark Matter Simulations: A Wavelet-Statistical View

[astro-ph:0804.0373] F. Schmidt et al.
Weak Lensing Effects on the Galaxy Three-Point Correlation Function

[astro-ph:0804.0382] R. U. Abbasi et al.
Search for Correlations between HiRes Stereo Events and Active Galactic Nuclei

[astro-ph:0804.0543] M. Schmalzl et al.
The Initial Mass Function of the Stellar Association NGC 602 in the Small Magellanic Cloud with Hubble Space Telescope ACS Observations

gravitational microlensing tutorial? [astro-ph:0803.4324]
Recent Developments in Gravitational Microlensing by A. Gould

paper with a very interesting title: [astro-ph:0803.3604]
Was There A Big Bang? by R. K. Soberman and M. Dubin

not astrostatistics but atypical statistical application, interesting topic, and good discussions:[stat.AP:0804.0079]
Statistical analysis of an archeological find by A. Feuerverger
Discussants are S.M. Stigler, C. Fuchs, D.L. Bentley, S.M. Bird, H. Höfling, L. Wasserman, R. Ingermanson, J. Mortera, P. Vicard, J.B. Kadane (Click names).

[ArXiv] 5th week, Jan. 2008

hlee — Fri, 01 Feb 2008 18:01:03 +0000

Some statistics papers were listed at the top, of which topics would interest some slog subscribers.

From statistics arxiv:

[stat.CO:0801.3387] Contemplating Evidence: properties, extensions of, and alternatives to Nested Sampling N. Chopin &C. Robert

[math.ST:0801.4329] Estimators of Long-Memory: Fourier versus Wavelets G. Fay et.al. (not comprehensible but the title is more than interesting)

From astro-ph:

[astro-ph:0801.4041] Quantifying parameter errors due to the peculiar velocities of type Ia supernovae R. Ali Vanderveld

[astro-ph:0801.4233] Effects of the interaction between dark energy and dark matter on cosmological parameters J. He & B. Wang

[astro-ph:0801.4889] Temporal variability and statistics of the Strehl ratio in adaptive-optics images S. Gladysz

[astro-ph:0801.4751] Low-Luminosity Gamma-Ray Bursts as a Distinct GRB Population:A Monte Carlo Analysis F Virgili, E Liang, &B Zhang
[astro-ph:0801.4759] Optical afterglow luminosities in the Swift epoch: confirming clustering and bimodality M. Nardini, G. Ghisellini & G. Ghirlanda

(The last two papers mentioned Kolmogorov-Smirnov test and probability)

Signal Processing and Bootstrap

hlee — Wed, 30 Jan 2008 06:33:25 +0000

Astronomers have developed their ways of processing signals almost independent to but sometimes collaboratively with engineers, although the fundamental of signal processing is same: extracting information. Doubtlessly, these two parallel roads of astronomers’ and engineers’ have been pointing opposite directions: one toward the sky and the other to the earth. Nevertheless, without an intensive argument, we could say that somewhat statistics has played the medium of signal processing for both scientists and engineers. This particular issue of IEEE signal processing magazine may shed lights for astronomers interested in signal processing and statistics outside the astronomical society.

IEEE Signal Processing Magazine Jul. 2007 Vol 24 Issue 4: Bootstrap methods in signal processing

This link will show the table of contents and provide links to articles; however, the access to papers requires IEEE Xplore subscription via libraries or individual IEEE memberships). Here, I’d like to attempt to introduce some articles and tutorials.

Special topic on bootstrap:
The guest editors (A.M. Zoubir & D.R. Iskander)^[1] open the issue by providing the rationale, the occasional invalid Gaussian noise assumption, and the consequential complex modeling in their editorial opening, Bootstrap Methods in Signal Processing. A practical approach has been Monte Carlo simulations but the cost of repeating experiments is problematic. The suggested alternative is the bootstrap, which provides tools for designing detectors for various signals subject to noise or interference from unknown distributions. It is said that the bootstrap is a computer-intensive tool for answering inferential questions and this issue serves as tutorials that introduce this computationally intensive statistical method to the signal processing community.

The first tutorial is written by those two guest editors: Bootstrap Methods and Applications, which begins with the list of bootstrap methods and emphasizes its resilience. It discusses the number of bootstrap samples to compensate a simulation (Monte Carlo) error to a statistical error and the sampling methods for dependent data with real examples. The flowchart from Fig. 9 provides the guideline for how to use the bootstrap methods as a summary.

The title of the second tutorial is Jackknifing Multitaper Spectrum Estimates (D.J. Thomson), which introduces the jackknife, multitaper estimates of spectra, and applying the former to the latter with real data sets. The author added the reason for his preference of jackknife to bootstrap and discussed the underline assumptions on resampling methods.

Instead of listing all articles from the special issue, a few astrostatistically notable articles are chosen:

Bootstrap-Inspired Techniques in Computational Intelligence (R. Polikar) explains the bootstrap for estimating errors, algorithms of bagging, boosting, and AdaBoost, and other bootstrap inspired techniques in ensemble systems with a discussion of missing.
Bootstrap for Empirical Multifractal Analysis (H. Wendt, P. Abry & S. Jaffard) explains block bootstrap methods for dependent data, bootstrap confidence limits, bootstrap hypothesis testing in addition to multifractal analysis. Due to the personal lack of familiarity in wavelet leaders, instead of paraphrasing, the article’s conclusion is intentionally replaced with quoting sentences:

First, besides being mathematically well-grounded with respect to multifractal analysis, wavelet leaders exhibit significantly enhanced statistical performance compared to wavelet coefficients. … Second, bootstrap procedures provide practitioners with satisfactory confidence limits and hypothesis test p-values for multifractal parameters. Third, the computationally cheap percentile method achieves already excellent performance for both confidence limits and tests.

Wild Bootstrap Test (J. Franke & S. Halim) discusses the residual-based nonparametric tests and the wild bootstrap for regression models, applicable to signal/image analysis. Their test checks the differences between two irregular signals/images.
Nonparametric Estimates of Biological Transducer Functions (D.H.Foster & K.Zychaluk) I like the part where they discuss generalized linear model (GLM) that is useful to expend the techniques of model fitting/model estimation in astronomy beyond gaussian and least square. They also mentioned that the bootstrap is simpler for getting confidence intervals.
Bootstrap Particle Filtering (J.V.Candy) It is a very pleasant reading for Bayesian signal processing and particle filter. It overviews MCMC and state space model, and explains resampling as a remedy to overcome the shortcomings of importance sampling in signal processing.
Compressive sensing. (R.G.Baranuik)

A lecture note presents a new method to capture and represent compressible signals at a rate significantly below the Nyquist rate. This method employs nonadaptive linear projections that preserve the structure of the signal;

I do wish this brief summary assists you selecting a few interesting articles.

They wrote a book, the bootstrap and its application in signal processing.

[ArXiv] 1st week, Dec. 2007

hlee — Fri, 07 Dec 2007 19:15:04 +0000

There’s only one day in the first week of December with no preprint appearance. Dubbing the week of Dec. 2nd as the first week is hoped to be accepted.

[stat.ML:0711.4983]
A Method for Compressing Parameters in Bayesian Models with Application to Logistic Sequence Prediction Models L. Li and R. M. Neal

[astro-ph:0711.4886]
Requirements on PSF Calibration for Dark Energy from Cosmic Shear S. Paulin-Henriksson et.al.

[astro-ph:0711.4895]
The impact of going beyond the Maxwell distribution in direct dark matter detection rates J. D. Vergados, S. H. Hansen, and O. Host

[astro-ph:0712.0003]
The Galaxy Cross-Correlation Function as a Probe of the Spatial Distribution of Galactic Satellites J. Chen

[stat.ME:0712.0283]
Wavelet methods in statistics: Some recent developments and their applications A. Antoniadis

[stat.ML:0712.0189]
Summarization and Classification of Non-Poisson Point Processes J. Picka and M. Deng

[astro-ph:0712.0588]
SZ and CMB reconstruction using Generalized Morphological Component Analysis J. Bobin et. al.

[astro-ph:0712.0610]
X-Atlas: An Online Archive of Chandra’s Stellar High Energy Transmission Gratings Observations O. W. Westbrook et.al.

[astro-ph:0712.0618]
Precision of Hubble constant derived using black hole binary absolute distances and statistical redshift information C. L. MacLeod and C. J. Hogan

[ArXiv] 5th week, Nov. 2007

hlee — Tue, 04 Dec 2007 00:58:58 +0000

Astronomers are hard working people, day and night, weekend and weekdays, 24/7, etc. My vacation delayed this week’s posting, not astronomers nor statisticians.

[astro-ph:0711.4356]
Transformations between 2MASS, SDSS and BVRI photometric systems: bridging the near infrared and optical S. Bilir et.al.

[astro-ph:0711.4369]
SED modeling of Young Massive Stars T. P. Robitaille

[astro-ph:0711.4387]
SkyMouse: A smart interface for astronomical on-line resources and services C.-Z. Cui et. al.

[stat.AP:0711.3765]
MCMC Inference for a Model with Sampling Bias: An Illustration using SAGE data R. Zaretzki et. al.

[astro-ph:0711.3640]
Large-Scale Anisotropic Correlation Function of SDSS Luminous Red Galaxies T. Okumura et.al.

[astro-ph:0711.4598]
Dynamical Evolution of Globular Clusters in Hierarchical Cosmology O.Y. Gnedin and J. L. Prieto

[astro-ph:0711.4795]
Globular Clusters and Dwarf Spheroidal Galaxies S. van den Bergh

[astro-ph:0711.3897]
Optical Monitoring of 3C 390.3 from 1995 to 2004 and Possible Periodicities in the Historical Light Curve
strong assumption on a Gaussian distribution. What would it be if the fitting is performed based on functional data analysis or Bayesian posterior draws? What if we relax strong gaussian assumption and apply robust estimation methods? It seems that modeling and estimating light curves seek more statistical touch!!!

[astro-ph:0711.3937]
Sequential Analysis Techniques for Correlation Studies in Particle Astronomy S.Y. BenZvi, B.M. Connolly, and S. Westerhoff

[astro-ph:0711.4027]
CCD Photometry of the globular cluster NGC 5466. RR Lyrae light curve decomposition and the distance scale A. A. Ferro et.al.

[astro-ph:0711.4045]
Fiducial Stellar Population Sequences for the u’g'r’i'z’ System J. L. Clem, D.A. VandenBerg, and P.B. Stetson

[astro-ph:0704.0646]
The Mathematical Universe Max Tegmark

[stat.ME:0711.3857]
Periodic Chandrasekhar recursions A. Aknouche and F. Hamdi

[math.ST:0711.3834]
On the Analytic Wavelet Transform J. M. Lilly and S. C. Olhede

[cs.IT:0709.1211]
Likelihood ratios and Bayesian inference for Poisson channels A. Reveillac

[astro-ph:0711.4194]
The Palomar Testbed Interferometer Calibrator Catalog G. T. van Belle et.al.

[astro-ph:0711.4305]
2MTF I. The Tully-Fisher Relation in the 2MASS J, H and K Bands Masters, Springob, and Huchra
Standard candle problems were realizations of various regression problems.

[astro-ph:0711.4256]
Observational Window Functions in Planet Transit Searches K. von Braun, and D. R. Ciardi

[astro-ph:0711.4510]
The benefits of the orthogonal LSM models Z. Mikulasek