Data Acquisition and Quality Control

Stations from nearly all networks operated by the BSL transmit data continuously to the BSL facilities on the UC Berkeley campus for analysis and archive. In this chapter, we describe activities and facilities which cross-cut the individual networks described in Chapters 3 - 8, including the facilities in McCone Hall, procedures for data acquisition and quality control, sensor testing capabilities and procedures, and a collaborative experiment in early warning.

While some of these activities are continuous from year to year, we have identified changes or activities which are specific to 2002-2003.

**Figure 9.1:** Data flow from the BDSN, NHFN, MPBO, HRSN, and BARD network into the BSL central processing facility.
$\begin{figure}\begin{center} \epsfig{file=acq_dataflow.eps,width=12cm}\end{center}\end{figure}$

McCone Hall Facilities

Because of the mission-critical nature of the automated earthquake processing, most computer systems operated by the BSL run on circuits with both UPS and generator power. Air conditioning is provided through both "building air" and a separate room AC unit.

Power

While the failure of the generator at Byerly Vault was traced to PPCS human error (the generator had been left in a mode where it would not automatically start when power was lost), the failure of the McCone generator was due to poor maintenance. Similar to the situation in 1999, it failed due to problems in the power system combined with a leak in the water pump.

Last fall, BSL staff met with Eric Haemer, Sara Shirazi, and several others from PPCS to discuss maintenance and routine load testing of the McCone generator. As a result, the McCone generator is scheduled for quarterly load tests and bi-monthly run tests.

These quarterly load tests have proven extremely valuable. In January 31st, 2003 test, the generator failed. The failure was due to a problem with the thermostat (since replaced), but the test also revealed that the BSL is drawing more AC power than desirable from the generator, largely from the growth of the computing facilities.

In order to reduce the load on generator and UPS, BSL staff were forced to remove computer systems to building power. Mission-critical systems (communications, data acquisition, data processing, and archiving) were kept on the generator and UPS circuits, which research-specific systems were migrated to building power. To accomplish this, building power circuits were added to the computer server room (the room had originally been designed with only generator and generator/UPS circuits).

This change has meant that BSL researchers have had to address the impact on their programs with long runtimes. Without UPS power, the servers and workstations will immediately shut off during a power failure, causing all active programs to terminate. BSL researchers have been asked to build or modify their programs to save incremental results to disk, in order to minimize the loss of work.

Air Conditioning

In parallel with power problems, the BSL has faced cooling problems in room 237 in the past year. As with power, the growth of the computing systems in the past year has led to an increased heat load. This came to a crisis during the fall of 2002, with peak temperatures in the computer room exceeded 85 $\deg$ when the AC unit failed. After consideration of several options, the BSL decided to add an additional AC unit to room 237. The new unit (which is not supported by UPS/generator power) has helped keep systems running this spring and summer, although the real test will be this fall.

In addition, the BSL staff set up a temperature monitoring system for room 237. For several years, we have relied on a temperature sensor within one our disk drives to notify us of excessive heat. In the last year, we purchased two temperature probes and a simple digitizing system that allows us to monitor temperature in several locations within the room. Complementing our pager notification, we can now monitor the temperature through real-time graphs accessible through the BSL Web site.

New Facilities

The BSL is actively working with the campus to relocate the critical operations of data acquisition, processing, archiving, and distribution to a more robust facility. With assistance from the Office of the Vice Chancellor for Research, the BSL has been granted space in a building currently under construction. The building is designed to current codes and has been given special attention for post-earthquake operations. Anticipated occupancy is in FY 2004-2005.

Data Acquisition

Central-site data acquisition for the BDSN/NHFN/MPBO is performed by two computer systems located at the BSL (Figure 9.1). These acquisition systems are also used for the Parkfield-Hollister electromagnetic array and for the BARD network. A third system s used primarily as data exchange system with the USNSN receives a feed from CMB, HUMO, MOD, SAO, and WDC from the the NSN VSAT. This system transmits data to the USNSN from HOPS, CMB, SAO, WDC, and YBH. Data acquisition for the HRSN follows a more complicated path, as described in Chapter 5.

Data acquisition and communication with the Quanterra data loggers depends both on the software on the recording systems and at the central site.

**Figure 9.2:** Dataflow in the REDI processing environment, showing waveform data coming in from the Quanterra data loggers (Q) into `comserv`. From comserv, data are logged to disk (via `datalog`), distributed to other computers (`mserv`), fed into the CDA for REDI processing, and spooled into a trace ring for export.
$\begin{figure}\begin{center} \epsfig{file=proc_dataflow.eps,width=12cm}\end{center}\end{figure}$

Comserv

The BSL uses the comserv program for central data acquisition, which was developed by Quanterra. The comserv program receives data from a remote Quanterra data logger, and redistributes the data to one or more comserv client programs. The comserv clients used by REDI include datalog, which writes the data to disk files for archival purposes, cdafill, which writes the data to the shared memory region for REDI analysis, and other programs such as the seismic alarm process, the DAC480 system, and the feed for the Memento Mori Web page (Figure 9.2).

The two computers that perform data acquisition also serve as REDI processing systems. In order to facilitate REDI processing, each system maintains a shared memory region that contains the most recent 30 minutes of data for each channel used by the REDI analysis system. All REDI analysis routines first attempt to use data in the shared memory region, and will only revert to retrieving data from disk files if the requested data is unavailable in the shared memory region.

Most stations transmit data to only one or the other of the two REDI systems. The comserv client program cs2m receives data from a comserv and multicasts the data over a private ethernet. The program mcast, a modified version of Quanterra's comserv program, receives the multicast data from cs2m, and provides a comserv-like interface to local comserv clients. This allows each REDI system to have a comserv server for every station.

We have extended the multicasting approach to handle data received from other networks such as the NCSN and UNR. These data are received by Earthworm data exchange programs, and are then converted to MiniSEED and multicast in the same manner as the BSL data. We use mserv on both REDI computers to receive the multicast data, and handle it in an identical fashion to the BSL MiniSEED data.

FIR Filter Changes

At 5:00 PM PST June 30th (July 1, 00:00 UTC), 2003, the BDSN and MiniPBO Q4120 Quanterras were reconfigured and rebooted to change the FIR filter for the 100 Hz channels from acausal to causal. The affected stations are: BDM, CVS, FARB, HUMO, OHLN, PACP, POTR, SBRN, and WENL. The new BDSN Q4120 station MNRC was upgraded on the 29th, since it is a new station and continuous telemetry has not yet been installed.

This change means that all Q4120/Q730 dataloggers operated by the BSL will use causal filters for sampling rates of 100 Hz and higher (the HRSN and NHFN have traditionally used causal filters for the higher sampling rates). Lower data rates will continue to use the acausal filters. This change does NOT apply to the BDSN sites with Q680/980 dataloggers, as the FIR filters are set in firmware and are not readily changed.

This change is motivated by the desire to improve phase picking on the 100 Hz channels. A detailed comparison of casual and acausal FIR filters and their effect on the data is available by Bob Uhrhammer and Bob Nadeau is available at http://quake.geo.berkeley.edu/bdsn/FIR_FILTRATION.pdf.

Seismic Noise Analysis

BSL seismic data are routinely monitored for state-of-health. An automated analysis is computed weekly to characterize the seismic noise level recorded by each broadband seismometer. The estimation of the Power Spectral Density (PSD) of the ground motion recorded at a seismic station, provides an objective measure of background seismic noise characteristics over a wide range of frequencies. When used routinely, the PSD algorithm also provides an objective measure of seasonal and secular variation in the noise characteristics and aids in the early diagnoses of instrumental problems. A PSD estimation algorithm was developed in the early 1990's at the BSL for characterizing the background seismic noise and as a tool for quality control. As presently implemented, the algorithm sends the results via email to the engineering and some research staff members and generates a bargraph output which compares all the BDSN broadband stations by components. A summary of the results for 2002-2003 is displayed in Figure 3.3.

Three years ago, we expanded our use of the weekly PSD results to monitor trends in the noise level at each station. In addition to the weekly bar graph, additional figures showing the analysis for the current year are produced. These cumulative PSD plots are generated for each station and show the noise level in 5 frequency bands for the broadband channels. These cumulative plots make it easier to spot certain problems, such as failure of a sensor. In addition to the station-based plots, a summary plot for each channel is produced, comparing all stations. These figures are presented as part of a noise analysis of the BDSN on the WWW at http://seismo.berkeley.edu/bdsn/psd/.

Sensor Testing Facility

The BSL has set up an instrumentation test facility in the Byerly Seismographic Vault in order to systematically determine and to compare the characteristics of up to eight sensors at a time. The test equipment consists of an eight-channel Quanterra Q4120 high-resolution data logger and a custom interconnect panel that provides isolated power and preamplification when required to facilitate the connection and routing of signals from the sensors to the data logger with shielded signal lines. Upon acquisition of the 100 samples-per-second (sps) data from the instruments under test, PSD analysis and spectral phase coherency analysis are used to characterize and compare the performance of each sensor. Tilt tests and seismic signals with a sufficient signal level above the background seismic noise are also used to verify the absolute calibration of the sensors. A simple vertical shake table is used to access the linearity of a seismic sensor.

The sensor testing facility of the BSL is described in detail in the 2001-2002 Annual Report.

**Figure 9.3:** Photo of the three Guralp CMG-1TD OBS units (serial numbers T1046, T1047 and T1055 seismometers in the Byerly Vault (BKS). Shown are the various circuit boards on the sides and the top of the sensor package. Three of the nine batteries used by the leveling system are on the left side, the system clock is on the circuit board on the front, the power board is on the right, and the 24-bit digitizer is on the top of each seismometer. The seismometers are in the mu metal shielded container mounted on leveling gimbals in the center.
$\begin{figure}\begin{center} \epsfig{file=bob03_keck_1.epsi, width=7cm}\end{center}\end{figure}$

**Figure 9.4:** Closeup of the portion of the power board containing the failed capacitor (just to the right of center).
$\begin{figure}\begin{center} \epsfig{file=bob03_keck_2.epsi, width=7cm}\end{center}\end{figure}$

Sensor Testing in 2002-2003

CMG-1T Ocean Bottom
Seismometers

Prior to the scheduled deployment of three CMG-1TD OBS sensor packages (Figure 9.3) on the ocean floor off of Washington State, beginning in the summer of 2003, we did extensive testing to verify the operation of the seismometers and the wide range leveling system, to verify the calibration of the seismometer and to characterize the background noise PSD performance of the seismometers. The following paragraphs provide a synopsis of the various problems we encountered when testing the three OBS systems. These problems significantly delayed the testing of the sensors and the lab personnel consequently spent more time on this project than was initially anticipated.

The testing of the OBS system in the Byerly Seismographic Vault (BKS) started on January 14, 2003 with the arrival, unpacking and installation of the three CMG-1TD seismometers (serial numbers T1046, T1047 and T1055) on the seismic pier in the Byerly Seismographic Vault (BKS). We installed a V200 FRAD in order to have sufficient serial ports to telemeter data the three sensors under test back to the lab. The next day we installed four 12 volt batteries (UPS12-310 type) to provide separate power for each of the three OBS systems and to the GPS clock. When we powered up T1047, a capacitor on the power input board caught fire and burned up with a spectacular flash and smoke within a second of applying power (see Figure 9.4). We confirmed that the power polarity was correct and we suspected that the polarized power capacitor was installed backwards. T1046 powered up without problems and responded to commands. T1055 was left unpowered pending inspection by Digital Technology Associates (DTA) (the US distributor for Guralp). DTA replaced the defective power board on T1047 we tested the unit and found it to be within specifications. The three GPS clock modules associated, one for each OBS system, were tested one at a time on the front of the BKS vault. All three GPS clocks tested good so we installed one of the clocks on top of the entrance to the vault and ran cabling back to provide time to the three OBS sensors. DTA replaced the defective power board on T1047 we tested the unit and found it to be within specifications. Plastic bags were placed over the OBS units to to keep dust and breezes off of the exposed sensors and electronics.

**Figure 9.5:** Custom made titanium pressure spheres which are designed for deployment at depths of up to 3.5 km. The hemispheres were made using an injection molding process. There are two access ports drilled into the flat top of each sphere, one for the penetrator containing the wiring cable and one for purging with argon gas (and for relieving any pressure differences so that the hemispheres can be separated). The handle on top is designed for the Remote Operated Vehicle (ROV) which deploys the sphere on the sea floor.

**Figure 9.6:** Closeup of the OBS installed in the lower hemisphere of the pressure vessel. Shown are the various circuit boards on the sides and the top of the sensor package. The thumb is pointing towards the rechargeable batteries which are used to to supply the extra current required by the two high torque motors used in the leveling system. On the right side of the sensor is the circuit board containing the internal clock. On the left front is the the I/O circuit board and on top is the 24-bit digitizer. The seismometers are in the mu metal shielded container mounted on leveling bowl. Also visible around the inside of the hemisphere, and just below the flange, is the high density foam insulation.
$\begin{figure}\begin{center} \epsfig{file=bob03_keck_4.epsi, width=7cm}\end{center}\end{figure}$

**Figure 9.7:** Power Spectral Density (PSD) plot of the background and instrumental noise levels in dB as a function of period. Shown are the PSD's for T1047 and T1055. The PSD for the co-sited STS-1 Z and the low seismic noise model are shown for reference. Note that the PSD for T1055 is excessively noisy at periods longer than $\sim$ 10 seconds.
$\begin{figure}\begin{center} \epsfig{file=bob03_keck_5.epsi, width=7cm}\end{center}\end{figure}$

We encountered problems in telemetering data back to the lab so we set up a laptop with the SCREAM software provided by Guralp to locally log data from the sensors and troubleshoot the systems. We encountered no errors that would indicate telemetry problems when logging the data locally. When the three OBS's recorded a local earthquake, we discovered that horizontals on T1047 have half gain and inverted polarity. This was raw data so software and transfer functions errors are excluded. We found that the onboard rechargeable batteries on T1047 and T1055 were not charged enough to lock the seismometers. We moved the good power board from T1046 in order to lock the T1047 and T1055 seismometers. All three power boards were then removed and sent back to Guralp for repair with an expected turnaround time of three weeks.

On March 24th the "improved" power boards were installed on the three OBS units and the revised firmware was successfully uploaded to the onboard digitizers. During the initial simultaneous testing of all three OBS sensors, we found that the power management has improved and seems to work properly and also that the traces from all three sensors, as recorded on a local laptop computer, looked coherent in amplitude and phase. We then connected the three OBS units to the telemetry link back to the lab for further testing. The azimuth command was tested on T1055 by locking the sensors, rotating it 90 degrees counter clockwise and then using the azimuth command to reconfigure for NS/EW orientation. However, we found that T1055 would not re-level. Subsequently T1946 was removed for testing and evaluation by DTA. T1046 had some gaps in the telemetered data so we swapped its telemetry to a different serial port on the FRAD to see if a good OBS system produces data gaps through the same serial port.

On April 18th, T1046 was returned from Guralp and reinstalled in the BKS vault. Over the next couple of weeks, the OBS sensors were sequentially tested with local recording on the laptop computer. All three OBS units are operating nominally within specifications and we await the arrival of the titanium pressure vessels from MBARI to complete the testing.

On June 20th, the MBARI crew arrived with the pressure vessels (Figure 9.5) and we spent most of the day installing the OBS units within the pressure vessels (Figure 9.6). The MBARI crew had installed $\sim$ 0.5 inch thick high density foam insulation in the upper titanium hemisphere and $\sim$ 4 inches into the lower hemisphere to inhibit convection within the enclosed pressure vessel. Additionally, each pressure vessel was purged with argon gas to further inhibit convection within the titanium pressure vessels. T1046 and T1055 were reconnected to the telemetry back to the lab and T1047 was taken to MBARI for testing in their cold room to determine whether or not the internal clock on the OBS unit met the factory specifications when operated at 4 degrees Celsius (the nominal temperature of the water on the ocean floor).

Subsequent testing indicated that T1055 had a high Z-component noise level and we suspected that it had drifted off center. We successfully recentered both OBS units via the laptop computer. T1046 was now operating nominally within specifications but T1055 remains noisy (Figure 9.7) so it will require further testing. The OBS units we picked up by the MBARI crew for transporting to the University of Washington on July 15th.

UrEDAS Project

The established joint notification system in Northern California provides accurate and reliable determination of earthquake parameters, but there is a time delay between the occurrence of an event and the determination of its size. In an emergency, this time delay prevents actions which could mitigate damage from strong ground shaking. In an effort to develop such capability with the BDSN, we started an experiment collocating a set of UrEDAS (Urgent Earthquake Detection and Alarm System; see Nakamura, 1996), an integrated real-time earthquake warning system, with the BDSN site BKS in 2001. Previous annual reports have described the UrEDAS system and its installation in Byerly Vault. Here we provide an update.

Collocating Experiment

The initial system installation at BKS was completed with the event detection and notification in February 2001 and was upgraded to transmit waveform data to the BSL in July 2001. The SDR crew visited the site in July 2002 to check on the equipment and to revise the values of the parameters used by the UrEDAS algorithms. They again visited the site in July 2003 to upgrade the software and revise the values of the processing parameters that determine the seismic wave apparent azimuth and dip. Figure 9.8 shows the UrEDAS sensors and Figure 9.9 shows an illustration of the UrEDAS network configuration.

**Figure 9.8:** UrEDAS sensors installed at BKS site. The dedicated PC-based processing system (not shown here) is located in an adjacent room.
$\begin{figure}\begin{center} \epsfig{file=bob03_uredas_11.eps,width=8cm}\end{center}\end{figure}$

**Figure 9.9:** Schematical illustration of the UrEDAS collocation experiment with a Streckeisen STS-1 broadband instrument and a Kinemetrics FBA-23 strong motion accelerometer with a Quanterra Q980 data logger at BKS.
$\begin{figure}\begin{center} \epsfig{file=bob03_uredas_12.eps, width=8cm}\end{center}\end{figure}$

Rapid Event Detection

In the UrEDAS system the event detection velocity threshold is pre-set; the epicentral azimuth is estimated from the direction of the initial motion projected on the horizontal plane; and the preliminary estimate of the distance and magnitude is based on the frequency content and amplitudes of P-wave first motions ( $\sim$ 3 sec). An alarm can be issued if a hazardous earthquake is detected by P-waves (Version 1 E-mail). If an S-wave arrival is detected, the preliminary estimate is revised (Version 2 E-mail)

The epicentral distance (

) is estimated using the relation $log R = a \cdot log A + b \cdot log T + c \mbox{ }$ where

is the amplitude of the initial P-wave motion (in mkine),

its prominent period, and

and

are constant. The magnitude is estimated from the prominent period (

) of the initial P-wave motion using the relation $M=3.2 \cdot log T + 5.26$ . We do not suppose that these relations apply universally but are testing them empirically. A UrEDAS waveform example is shown in Figure 9.10 and an expanded view of the P-wave is shown in Figure 9.11. The azimuth determination shows systematic biases, most likely due to the nearby Hayward fault where the impedance can change by $\sim$ 40% across the fault zone. The erroneous location estimates can be also attributed to the propagation path effects through the faults, the near-site structural heterogeneity and/or noise level. If the azimuth estimate becomes reliable, the combined information from two stations could also make a reasonable estimate of an epicentral distance.

**Figure:** UrEDAS waveforms from a M 2.3 earthquake which occurred 32.2 km SE (123 $\deg$ azimuth) from Berkeley and 8 km NNW of Pleasanton, CA. The vertical scales are in milliKine (1 Kine is 1 cm/sec).
$\begin{figure}\begin{center} \epsfig{file=bob03_uredas_14.epsi, width=8cm}\end{center}\end{figure}$

**Figure 9.11:** Expanded view of Figure 9.10 showing the first few seconds of the P-wave. Note that the P-wave particle motion is linear for the approximately the first cycle and then it predominantly elliptical owing to the near-receiver structural complexity of the crust and the proximity to the Hayward fault zone.
$\begin{figure}\begin{center} \epsfig{file=bob03_uredas_15.epsi, width=8cm}\end{center}\end{figure}$

The estimated magnitudes of small local events in the epicentral distance between 20 and 200 km were within the range expected from other experiments. Magnitudes of the smaller events (M

2.5) tend to be overestimated, and those of events at farther distances (R

200 km) are underestimated. See the previous Annual Report for more detail.

Since the installation of the UrEDAS system in February 2001, there have been 575 UrEDAS paged events (1.3 $\leq$ M $\leq$ 8.1) and 387 of these had corresponding NCSN events (within a 500 km radius and with a theoretical P-wave onset time at BKS within 20 seconds of the UrEDAS detection time). The 188 uncorrelated UrEDAS events are a mix of teleseisms (which UrEDAS has a tendency to mislocate as local events), some small local events near Berkeley, and a few random noise triggers. The UrEDAS performance was evaluated by comparing the event parameters with those recorded in the ANSS composite catalog. The event detection performance was satisfactory, although UrEDAS is designed to detect primarily local events (R $\leq$ 200 km) and it does not have the ability to distinguish between teleseismic and local events at present.

We have done some preliminary comparison of the waveform data recorded by the BKS broadband instrument with those recorded by the UrEDAS. Because of the complexities of seismic structure, nonlinearities involved in the propagation of the complex faults areas, this problem does not lend itself to easy analysis without systematic and more advanced analyses and calibrations. We focus on improving the algorithm to rapidly evaluate preliminary earthquake source parameters, i.e., magnitude and location.

Discussion

To date UrEDAS readily detects the occurrence of local/regional events from the P-wave signal. It also does a fair job of determining the source distance out to 160 km or so but the azimuth determination is basically unusable. UrEDAS also has biased magnitude estimates. The UrEDAS algorithm assumes a one-dimensional velocity model with straight line propagation paths and a three-dimensional model of the crustal structure will likely be required to significantly improve the azimuthal estimates. Also, the magnitude estimation algorithm needs further tuning. During their July, 2003 visit, the UrEDAS engineers updated some of the UrEDAS algorithms parameter values. In particular, they shortened the time interval that is used to estimate the azimuth from the P-wave waveform.

Assuming that the primary goal is to determine the event location and size as rapidly as possible, the fastest approach will prove to be a hybrid approach where the remote stations determine the azimuth and ramp growth rate and associated uncertainties and the central site uses a fuzzy logic algorithm to determine the location and size of the event. The primary advantage of this hybrid method is that the ramp growth rate can be reliably determined before the S-wave arrives. In the limiting case, and with a sufficiently high station density, one could even go so far as to determine and report from the remote sites using only the broadband P-wave impulse, the associated azimuth and apparent angle of incidence (along with estimates of their resolution). The central site could then coalesce the data into a viable and rapid event report.

The critical issue for a successful installation of a UrEDAS type system in the BDSN is the calibration of specific site effects at individual stations. A joint use of the single station detection system with the current northern California earthquake notification system would significantly increase the capability of real-time earthquake warning system.

Acknowledgements

Doug Neuhauser, Bob Uhrhammer, Lind Gee, Pete Lombard, and Rick McKenzie are involved in the data acquisition and quality control of BDSN/NHFN/MBPO data.

Development of the sensor test facility and analysis system was a collaborative effort of Bob Uhrhammer, Tom McEvilly, John Friday, and Bill Karavas. IRIS and DTRA provided, in part, funding and/or incentive to set up and operate the facility and we thank them for their support.

Bob Uhrhammer led the testing and problem solving effort of the KECK sensors, with help from John Friday, Doug Neuhauser, and Bill Karavas.

Bob Uhrhammer and Bob Nadeau evaluated the impact of the FIR filters on the BDSN data.

Fumiko Tajima initiated the collaboration with SDR on testing the UrEDAS system, which is now coordinated by Bob Uhrhammer. Doug Neuhauser, Bill Karavas, John Friday, and Dave Rapkin helped with installation and maintenance. We thank Yutaka Nakamura and his colleagues at SDR for providing us with the installation of UrEDAS system and information on the accumulated data by this system.

Bob Uhrhammer, Lind Gee, and Doug Neuhauser contributed to the preparation of this chapter.

References

Nakamura, Y. and A. Saito, Train stopping system for the Tohoku Shinkansen (in Japanese), Proc. of Semiannual meeting of the Seismological Society of Japan, 82, 244, 1982.

Nakamura, Y., Real-time information systems for seismic hazards mitigation UrEDAS, HERAS, and PIC, Quarterly Report of Railway Technical Research Institute, 37, 112-127,1996.

Scherbaum, Frank. Of Poles and Zeros: Fundamentals in Digital Seismology, Volume 15 of Modern Approaches in Geophysics, G. Nolet, Managing Editor, Kluwer Academic Press, Dordrecht, xi + 257 pp., 1996.

Tapley, W. C. and J. E. Tull, SAC - Seismic Analysis Code: Users Manual, Lawrence Livermore National Laboratory, Revision 4, 388 pp., March 20, 1992.