EEG-fNIRS cookbook

What to keep in mind when combining Brain Products EEG with fNIRS

In this article, we will explore a series of concepts that are relevant when combining Brain Products EEG amplifiers with fNIRS. We start with some general concepts related to the features of the two techniques and then dive a bit deeper into how to structure your experiment, combine the sensors, and ensure synchronization between the two signals. Finally, we offer some suggestions on where to perform your data analysis.

At the end of this article, you will also find links to the other support articles in this EEG-fNIRS series, which focus on details that are relevant when combining Brain Products equipment with specific equipment of other NIRS manufacturers (see Tested combinations – Manufacturer-specific articles below).

Some manufacturers offer premade montages and/or software to generate your own montage (see Tested combinations – Manufacturer-specific articles below). Alternatively, you can search for software tools that allow you to create such montages, for example this MATLAB^®-based tool: https://github.com/DOT-HUB/ArrayDesigner.

Why EEG and fNIRS?

In the last twenty years, the combination of different neurophysiological technologies has attracted the attention of neuroscience researchers. As co-registration offers the possibility to examine cortical activity more comprehensively than with one modality alone, the field has observed an increase in multimodal measurement techniques. Amongst the many, simultaneous EEG and fNIRS has a series of functional and practical advantages. On the one hand, EEG directly measures the brain’s rapid electrical activity, arising from the summation of post-synaptic potentials, primarily of pyramidal neurons of the cortex, which are projected to the scalp, and detected in the form of a difference in potential by at least two electrodes. On the other hand, fNIRS is linked to the brain’s hemodynamic response and, specifically, localizes the slower changes in oxygen metabolism that follow neural activation, via sensors called optodes. In other words, EEG and fNIRS signals capture different events linked to the same neurophysiological activity and have complementary properties: EEG, being based on electrical signal projected to the scalp, has an exquisite temporal resolution (millisecond precision), but limited spatial resolution; fNIRS, being connected to changes in brain blood-flow has a good spatial resolution (<1cm), but limited temporal resolution (~3-6 seconds).

Figure 1. The plot shows temporal resolution (low to high) on the vertical axis and spatial resolution (low to high) on the horizontal axis. EEG is characterized by high temporal resolution but limited spatial resolution, whereas fNIRS shows complementary properties, with higher spatial resolution and lower temporal resolution.

Due to the distinct nature of their signals and detection methods, EEG and fNIRS signals do not interfere with each other. This characteristic simplifies the process of data recording and analysis compared to other modalities like fMRI and MEG, where managing interference poses significant challenges. While EEG measures scalp potentials, fNIRS uses light to measure cortical haemodynamic response. Despite the absence of interference between the two technologies, users should nevertheless keep in mind that EEG-fNIRS co-registrations are still subject to the same artifacts commonly discussed for individual (unimodal) recordings.

What to keep in mind when combining EEG-fNIRS

If you want to perform simultaneous EEG-fNIRS measurements, you will need to think about:

Experimental design
Montage and cap set-up
Checking the signals
Triggering, synchronization, and signal acquisition
Data acquisition
Data analysis

Each of these topics will be covered in the rest of this article.

1 Experimental design

An important point to consider is how to structure your experiment design. Due to the different natures of the signals, in unimodal recordings, you would probably opt for an event-related / trial design for EEG and for a block design for fNIRS. However, if you are combining the two modalities, you probably also need to combine the two types of design.

For example, in the case of a visual task, for the EEG part you might want to have event markers for each stimulus presentation to be able to perform an event-related analysis (e.g., ERP or time-frequency). For the fNIRS part, you will probably analyze the hemodynamic response for the entire experimental block, as defined by start and end markers. Figure 2 shows the scheme for a simplified EEG-fNIRS visual experiment.

Figure 2. The figure shows how to adapt a visual task (checkerboard task, see Sandmann and colleagues 2012) for a simultaneous EEG-fNIRS experiment. The red arrows mark the events for each trial (i.e., stimulus presentation), which are important for the event-related analysis of the EEG signal. The blue arrows mark the events for each presentation block, which are important for the fNIRS analysis.

Unsure about experimental design?
Check out the Brain Products Academy and sign up for our dedicated mailing list to hear about our upcoming EEG workshops & webinars!

2 Montage and cap set-up

Your specific montage is likely to be influenced by your research question and, for this reason, you might want to start thinking about your EEG-fNIRS montage during the planning phase of your study.

2.1 Defining your EEG-fNIRS montage

While EEG montages tend to cover the whole scalp (usually according to the 10-20 system; Jaspers et al., 1958), fNIRS montages usually focus on specific regions. When the two technologies are combined, EEG and fNIRS sensors might end up competing for the same locations. Deciding how the sensors should be arrayed on the same cap and which one should be privileged on specific hotspots will depend on your research question and might be guided by previous literature. You can create an EEG-fNIRS channel array by first defining the fNIRS montage according to the instructions of your manufacturer of choice and then complement it by selecting the EEG montage based on the same 10-20 system, or vice versa. For example, if you are planning a visual task, both EEG electrodes and NIRS optodes should be placed around occipital locations (O1, O2, Oz, etc.) to be as close as possible to the visual cortex (Figure 3).

Figure 3. The picture displays an occipital EEG-fNIRS montage, with 32 EEG electrodes and 8×8 visual cortex fNIRS montage, used for a visual task (checkerboard task, see Sandmann and colleagues 2012).

2.2 Select the right cap

Since EEG electrodes and NIRS optodes will compete for the same scalp locations, it is recommended to have a cap which can host both sensors and eventually place them close to each other in specific hotspots to ensure the best coverage possible. Moreover, because of the nature of the fNIRS signal, it is desirable for the cap to have a dark fabric to decrease unwanted optical reflection and improve signal quality.

In other words, we need a cap:

With a large number of slits/cuts
Whose slits/cuts can physically host both the EEG and the fNIRS holders
Whose fabric is black (because this colour seems to favour the performance of the fNIRS optodes)

Such a cap would be an actiCAP cap with 128 or even 160 slits and black fabric. It is manufactured by Easycap GmbH and can be ordered via our distributor network.

Figure 4. actiCAP cap in black fabric, with 128 slits, 32 EEG holders on the first 10-20 positions.

Depending on your fNIRS manufacturer of choice, other cap options might be available. Check the Tested combinations – Manufacturer-specific articles linked at the bottom of this article

2.3 Populate the cap

Now that you have defined your EEG-fNIRS montage and selected a cap with enough slits, you can start populating the cap with the sensor holders.

For the EEG part, you can have us pre-populate your cap with the snap holders. If you however, because of your specific montage, need to move a holder to another slit, you can do so by following the steps shown in Figure 5. Start by grabbing the fabric with one hand, with the other, pinch the holder ring, and gently pull it backwards, until you see some space in the slit. Then angle the snap holder upwards and slide it out of the slit. To place the holder to a new location, place the holder on top of the target slit. Gently pull the fabric, and slide the holder in, until it is surrounded but the edges of the slit.

EEG-fNIRS Cookbook - Figure 5
Figure 5. Populating the cap with the snap holders: a) identify the slit where you would like to place the holder; b) slide the holder through the slit from the inside; c) gently pull the holder through by making sure that the fabric embraces the rim of the holder; d) your holder is now in position!

Once the holders are in the right position, you can place the sensors into the holders (Figure 6). For the EEG part, populating the actiCAP snap cap follows the same procedure of any unimodal measurement (the procedure is described here or you can check the actiCAP setup and corresponding video).

EEG-fNIRS Cookbook - Figure 6
Figure 6. Snapping the actiCAP slim electrodes into the snap holders: a) grab the snap holder with one hand, and the corresponding actiCAP slim electrode with the other; b) slide the holder into the opening of the snap older; c) rotate the electrode to the decided position.

For the NIRS part, cap population depends on the specific implementation of each manufacturer (see Tested combinations – Manufacturer-specific articles below). In general, it will involve placing the NIRS holders according to your montage and then adding the optodes.

3 Checking the signals

Once the montage is defined and the cap is fitted, it is time for your participant to wear it. This procedure is the same as a normal EEG cap setup followed by an fNIRS cap setup. 

For the EEG, you may want to reduce the impedance of the actiCAP slim electrodes via BrainVision Recorder (click here for a detailed description or check this video).

For the NIRS, you can check signal quality within the specific NIRS software (see Tested combinations – Manufacturer-specific articles below).

4 Triggering, synchronization, and signal acquisition

In multimodal measurements, it is always crucial to have shared markers to identify the same events in the separate time-series in order to allow triggering and event synchronization for both online and offline analysis. For this reason, you will need to understand how to share events between EEG and fNIRS for task event marking and a precise time-series synchronization.

Depending on the system used, this can be achieved using:

Software makers via the Lab Streaming Layer (LSL) protocol (official website)
Shared hardware triggers

4.1 LSL: triggering, synchronization

LSL is an open-source communication protocol that allows the unified collection of data streams coming from different sources connected to the same network. In our case, the data stream sources will be the EEG amplifier, the fNIRS device, and the markers coming from a stimulus presentation software. LSL can make sure that the time series are synchronized and the markers are present in both.
To do this, follow these steps.

4.1.1 Create a network

For LSL to receive all the data streams, the computers to which the EEG and the fNIRS are associated, as well as the stimulus presentation computer, need to be connected to the same network and be mutually discoverable. For this, we recommend creating a private local network either via a wired ethernet connection or through an access point router which grants a stable Wi-Fi connection.

4.1.2 Connect the streams to LSL

The next step is connecting the data streams in LSL. This needs to be done separately for each the EEG stream, the fNIRS stream, and for the markers (i.e., stimulus presentation software). Depending on the manufacturer’s implementation, the LSL connection is either handled with a dedicated app or activated within the data acquisition software:

For Brain Products amplifiers, you can find an amplifier-specific app on our GitHub page that is downloadable from the Releases (e.g., LiveAmp LSL connector).
For the fNIRS see Tested combinations – Manufacturer-specific articles below for LSL instructions.
For the LSL markers coming from the stimulus presentation software, you have two options:
- With code – for example, in E-Prime^®, Presentation^®, MATLAB^®, or Python, you can check this resource
- Without code – use the TriggerBox Plus, which offers a one button solution to convert your existing triggers into LSL markers

4.1.3 Monitoring the LSL streams

The data streams will have to be monitored in separate software solutions:

For Brain Products amplifiers, you can use our freeware BrainVision LSL Viewer to monitor EEG and LSL markers.
For the fNIRS stream, you need to check the manufacturer’s instructions.

4.1.4 Recording the LSL streams

To record the LSL streams, there might be subtle variations depending on the specific manufacturer’s implementation of LSL:

For the EEG part, you can use LabRecorder, the original LSL acquisition app (LabRecorder.exe), which can acquire any type of LSL stream together with the markers in the same XDF file. (As described in this blog entry.) When you are ready to start your recording, open the app, click “Update”, select the streams of interest, your target directory, and hit start acquisition. See section 6 Data Analysis for more details on how to analyze XDF files.
For the fNIRS part, you can also use LabRecorder or manufacturer-specific software implementations.

(1)

(2)

(3)

Figure 7. Connecting, Recording, and Monitoring LSL streams. (1) Grabbing the LSL streams in BrainVision LSL Viewer; (2) Grabbing the LSL streams in LabRecorder; (3) Visualizing the EEG data and LSL marker streams (golden three-digit numbers in proximity of the x-axis).

4.2 Hardware triggers: triggering, synchronization, and acquisition

A classical way of having event markers in both systems is to use hardware triggers that are generated by a single common source. Depending on your paradigm, your event markers might be generated by a presentation software (via USB port) or another hardware source (e.g., an LPT or BNC port from a response box). You can use the TriggerBox Plus to receive event markers from different sources and forward them, with millisecond precision, to target devices. In our case, the target devices are the EEG amplifier and the fNIRS. For this reason, we need a way of sending the event markers to both.

4.2.1 Trigger mirroring

Our recommended solution is to use the trigger mirroring function, which is available on actiCHamp Plus and LiveAmp with Sensor and Trigger Extension. It’s easy! Just connect the TriggerBox to the trigger-in port of the EEG amplifier, connect the DSUB-9 F trigger-out to the trigger-in port of the fNIRS device, and activate the trigger mirroring in Recorder (Figure 8). Your event markers will be automatically forwarded from the EEG to the fNIRS device, without any significant delay, given the slower sampling rate of the fNIRS.

EEG-fNIRS Cookbook - Figure 8
Figure 8. Trigger Mirroring implementation in Recorder. Left: for actiCHamp Plus. Right: for LiveAmp with Sensor & Trigger Extension.

4.2.2. Wireless Trigger (only for LiveAmp with STE)

As described here, our Wireless Trigger for LiveAmp STE can send triggers from one transmitter to multiple receiving units. Now, if your NIRS system has a LPT trigger-input port, it can be equipped with a Wireless Trigger Receiver. All you need is an adapter cable (DSUB9 to LPT) and a USB to DC power adapter (available at Easycap: KB-USB-DC0,65) and a USB power bank.

4.2.3. Separate trigger replicator

If your Brain Products amplifier does not have the trigger mirroring function, or if you simply prefer not to use it, you can also connect the TriggerBox to a trigger replicator or a Y-cable, via a DSUB25-to-LPT cable.

EEG-fNIRS Cookbook - Schema
Figure 9. Schematic drawing of an EEG-fNIRS set up.

Note: The Amp port of the TriggerBox Plus, uses a modified version of the traditional LPT port, which we call DSUB25. As described in the TriggerBox Plus Operating Instructions, the main difference is that the Pin 24 of the Amp port of the TriggerBox supplies 5V. Using a traditional LPT-to-LPT cable or a modified DSUB25-to-SUB25 does not allow communication between the TriggerBox and devices with a traditional LPT port, such as the NIRx Parallel Port Replicator. In order to enable such compatibility, as of November 2019, TriggerBox kits are delivered with a DSUB25-to-LPT cable. If you already have a TriggerBox, please check with your distributor whether you have the correct cable.

5 Data acquisition

The way you will acquire data is dependent on your triggering and synchronization strategy (see section 4).

If you are using LSL markers, see Section 4.1.4.

If you are using hardware triggers, you can use BrainVision Recorder to acquire the EEG signal and store it in the BrainVision format (.eeg, .vhrd, .vmrk) and your fNIRS software of preference.

6 Data analysis

Once both EEG and fNIRS data have been recorded together with the same LSL markers, it is possible to analyse them separately with your favourite data analysis software.

EEG data can be analyzed with BrainVision Analyzer directly if acquired with BrainVision Recorder or after a conversion if recorded in XDF via LabRecorder / LSL (see this article for instructions how to do this).

fNIRS data can be directly analyzed with your platform of preference if stored in a directly readable format. Alternatively, similar to EEG data, the file can be extracted from the XDF file, converted, and exported to one of the aforementioned analysis software solutions.

If you want to perform an analysis on the same platform, you can consider BrainStorm, MNE or FieldTrip.

Tested combinations – Manufacturer-specific articles

The implementation of points (2), (3), (4) and (5) need to take into account the specific features of the devices you are planning on combining. For this reason, you can refer to the dedicated support article for your set up:

Application	EEG hardware	NIRS hardware	Synchronization type
Mobile	LiveAmp actiCAP snap cap actiCAP slim electrodes TriggerBox Plus Sensor and Trigger Extension*	Artinis Brite (Artinis neoprene headcap)	LSL
Mobile	LiveAmp actiCAP snap cap actiCAP slim electrodes TriggerBox Plus Sensor and Trigger Extension*	Cortivision Photon (Cortivision Photon cap)	LSL
Mobile	LiveAmp actiCAP snap cap actiCAP slim electrodes TriggerBox Plus Sensor and Trigger Extension* DSUB9 to LPT cable* Wireless Trigger**	NIRx NIRSport2	LSL Trigger Mirroring via STE *Trigger Duplicated via Wireless Trigger
Stationary	actiCHamp Plus actiCAP snap cap actiCAP slim electrodes TriggerBox Plus	Artinis Brite (Artinis neoprene headcap)	LSL
Stationary	actiCHamp Plus actiCAP snap cap actiCAP slim electrodes TriggerBox Plus DSUB9 to LPT cable	NIRx NIRStar	Trigger Mirroring

Conclusion

We hope you have found this content helpful and that you now feel equipped and ready to start with your EEG-fNIRS experiments. Do not forget to cite us in your next publication (how to cite us) so we can see how you are using combined EEG & fNIRS for your experiment.

Please do not hesitate to contact us or your local distributor for more information or to answer remaining questions specific to your needs.

PostedApril 22, 2024

UpdatedMay 21, 2024

Tags: