Measuring moisture with an NIR spectrometer: beginner steps from power-on to readings

Spent the weekend messing with that near-infrared spectrometer borrowed from the lab, and hit quite a few pitfalls. My advisor told me to try out this Xinjie Guangyuan setup, saying it could be useful later for measuring water content in crude oil.

First, what this thing actually does. Near-infrared spectroscopy is basically shining a beam of light with wavelengths between 780 and 2500 nanometers at a sample and seeing which wavelengths get absorbed. Hydrogen-containing chemical bonds, like C-H, O-H, N-H, absorb in this band, and different substances absorb at different positions and strengths—like a fingerprint. So you first take a batch of samples with known composition and map "what the spectrum looks like" to "what the composition is." Then for unknown samples, you just scan once and it can calculate the composition. Fast—results in under a minute, no sample prep, and it doesn't destroy the sample.

What I used it for was water cut in oil-water two-phase flow. Crude oil coming from the oilfield is mixed with water, and you need to know how much water there is. I prepared 41 samples, water cut from 0 to 100%, in 2.5% steps, in 10 mm path length cuvettes.

From power-on to results

1. Power on and warm up. Ambient temperature 18 to 28°C, humidity 30% to 70%, and don't let direct sunlight hit the sample compartment. After powering on, wait about half an hour for the light source to stabilize.

2. Run self-test. Click instrument self-test in the software; it compares the measured spectrum against a standard spectrum and checks signal-to-noise ratio, energy, and wavenumber deviation. Don't skip this step—I skipped it the first day and all my spectra afterward were skewed.

3. Collect background. With the sample compartment empty, click collect background and save it. The background is "the optical path state with no sample," and every sample spectrum afterward gets subtracted by it.

4. Load the sample. Fill the cuvette to about two-thirds. Oil-water mixtures separate on standing, so shake well before measuring.

5. Collect spectrum. Set integration time and number of averages, and first click preview to see the real-time curve. Don't let the energy bar max out, and don't let it get so low you can barely see it—adjust the attenuator or integration time. When it's right, click collect sample and put the water cut in the filename.

6. Build the model. Import the spectra and corresponding water cuts into chemometrics software. I used PLS, partial least squares, which finds the directions in the spectrum most correlated with water cut. Hold out some samples for validation—don't train on all of them.

7. Get results. Once the model is built, scan a new sample and the software directly gives the predicted water cut.

Where the pitfalls are, and is it worth it

The first pitfall is bubbles. One tiny bubble in the cuvette shows up as a big bump in the spectrum, and the water cut can be off by several points. After loading, look at it against the light; if there's a bubble, tap the cuvette wall.

The second pitfall is fingerprints on the outside of the cuvette. Oil from your fingers on the optical surface is basically an extra layer of sample. Wipe twice with lens paper every time.

The third pitfall is background drift. The light source heats up after running a while, and the background shifts with it. From what I measured, re-collecting the background every twenty minutes is more stable.

The fourth pitfall is the most insidious: measuring before the oil and water are properly mixed. Oil on top, water on the bottom—the spoonful you take has a water cut that's nothing like the label, and no matter how you build the model it won't be accurate.

The advantages: fast, under a minute; no sample prep, non-destructive; one spectrum can compute several indicators at once; and it can be hooked up to a production line for online analysis.

The disadvantages too. It's an indirect method, entirely dependent on the model. If the model is poorly built, the numbers look plausible but are actually way off. Change the oil type or the temperature, and the model may need to be rebuilt. The upfront work of preparing samples and calibrating composition can't be skipped at all.

My judgment: for scenarios with large sample volumes and repeated measurement of the same indicator, it's very worth it. For measuring once or twice, or where sample composition changes daily, you're better off sending it to the lab for chemical analysis.

After learning this, my next step is to try the effect of temperature on the spectrum. Scan the same sample at different temperatures and see if the model drifts. If any of you have done similar calibration, how do you usually control for temperature?