Journal of Neurology and Neuroscience

Keywords

Hyperoxia; Traumatic brain injury; Mortality; Neurological outcomes; Intensive care; Mechanical ventilator.

Introduction

Language deficits, including impairments in comprehension and reading, are often present in early Alzheimer’s disease (AD) [1]. Given the complex relationship between language and other cognitive domains, difficulties with language comprehension and expression affect many areas of daily living and are important contributors to social exclusion [2]. Impairments in reading comprehension affect many functional activities, such as taking in news from a newspaper, understanding a book or email, or even working with a computer. Reading difficulties are strongly related to attention and executive function deficits [3]. Since attention is one of the earliest non-memory domains affected in AD [4] interventions targeting attention may improve functional areas, such as reading, in people with mild AD.

There is a breadth of behavioral interventions for improving cognitive function both in older adults without known disabilities and in adults with AD. Reviews of intervention practices for cognition in older adults have usually demonstrated domainspecific improvements [5-7] such that training in one cognitive domain (e.g., working memory) does not transfer to untrained domains (e.g., episodic memory) [8]. According to two systematic reviews, training that targets multiple cognitive domains for people with AD has mixed results, with some studies demonstrating improvements and others showing no functional change [9,10]. With advancements in technology, computerbased cognitive training approaches that feature interactive components for participants (e.g., video-gaming elements) have been proposed as a possible intervention modality [11-13]. Historically, cognitive interventions train and measure behavioral responses as indices of cognition. A major limitation of current behavioral interventions is the lack of real-time neurophysiologic metrics to objectively measure and guide the user’s learning.

Brain-computer interface (BCI) is a technology that has been developed to enhance, restore or replace physical or cognitive functioning using real-time invasive or noninvasive recording of brain signals as an input method to control the technology [14]. For example, the sensorimotor rhythm has been used in BCIs for control of computers, robotic arms, and wheelchairs by individuals with tetraplegia [14]. While much BCI research has involved healthy adults, a major clinical benefit of BCI is for people with disabilities [15] who may directly benefit from BCIs designed to restore or replace physical function. For individuals with severe speech and physical impairments secondary to incomplete locked-in syndrome, spelling BCIs have been designed to use the P300 signal, an endogenous eventrelated potential following a salient stimulus, as a ‘keystroke’ or intended selection for communication [16,17]. In this paradigm, target letter presentations are interspersed with non-target presentations. Perception of the salient target results in an attentional neurophysiological event-related potential (P300) to the intended target. This task requires the user to employ many cognitive skills, including (but not limited to) sustained visual attention to the screen, remembering the target throughout the task (working memory), selective attention to the targets on the screen amidst non-targets, and divided attention between the presented letters and feedback. As such, training with a P300 spelling paradigm may be beneficial across multiple cognitive domains.

Neurofeedback (NFB) is the delivery of real-time electroencephalography (EEG) based feedback. Few clinical NFB present cognitive tasks; most simply provide feedback to ongoing EEG frequencies such as alpha or theta band power [18,19]. Some NFB studies have used very simple recordings, e.g., 2-channel forehead strap systems that may be providing feedback based on the frequency of scalp muscle activity rather than brain activity [20]. Other NFB systems may be giving NFB based on simple alertness [21] which is known to affect ERPs [22] and BCI performance [21]. To improve cognitive performance, we believe use of NFB during performance of a cognitive task is better than using a simple rest state and thus there has been some merging of NFB and BCI research. This merging of BCI and NFB is also relevant, since training people to optimally use a BCI system may require NFB during BCI use.

While the P300 may be used to guide the BCI’s decision-making process for inferring the user’s intention, most P300 speller systems require too many stimulus presentations in order to generate the P300, and thus cannot not provide timely NFB. One group has used presentation of the chosen letter during use of a P300 speller as feedback, that improve performance at high letter flash rates, but this behavioral feedback is not quite the same as NFB [23]. The same group followed up with similar correct letter feedback and demonstrated changes in alpha activity at Pz consistent with greater attention during feedback training compared to no-feedback training [24]. As such, utilizing some EEG band measure activity may be a way to improve attention to BCI cognitive demands. This study used alpha activity for NFB since it has been used previously for NFB during tasks [25,26] and we had some preliminary data (see below in Calculation of Neurofeedback section). Training with EEG-based NFB to enhance attention is an important research direction in cognitive training [27]. There is some evidence of benefits of this training delivery method, including increased visual attention in young adults without disabilities [24] and improvement of attention deficit hyperactivity disorder symptoms in children [28]. In terms of aging, most studies based on recent systematic reviews have focused on healthy older adults [29,30]. One randomized wait-list controlled study of adults without disabilities (n=39), including 24 half-hour sessions of BCI training using a Stroop task, noted slight improvements on the Repeatable Assessment of Neuropsychological Status [31]. A larger follow-up study by the same group in 240 health elderly did not demonstrate clear improvements in the same cognitive outcomes but did demonstrate a sex moderation effect that suggested men improved more with the intervention than those men in the waitlist control with female participants showing no significant difference [32]. The limited number of studies included in systematic reviews that involved people with cognitive impairments (mild cognitive impairment or AD) all had significant weaknesses; inclusion of NFB as one component of a more general lifestyle intervention [33]; use of only a single channel of EEG [34]; lack of a control group or multiple baseline assessments to control for learning effects [26,35,36]; and use of a simple retrospective analysis of a poorly-characterized group of people of mixed ages with cognitive dysfunction treated with NFB [37]. One higher-quality paper studied 65 women with amnestic mild cognitive impairment and 54 control women who received NFB in a gamified interface [25]. There were two control groups: a treatment as usual (passive control) and an exergame done as frequently as the NFB sessions (active control). The NFB and the active control group both demonstrated improvements in some, but not the same, cognitive outcomes.

The aim of this feasibility study was to adapt an existing BCI system, BciPy [38], (available from https://github.com/CAMBItech/ BciPy), to also provide NFB and explore its application to facilitate improvement in attention and reading skills in people with mild AD. Outcome measures targeted skills within the domain of the proposed intervention, including selective attention to letters, speed of processing while reading, and working memory while manipulating letters. We hypothesized that NFB training would improve attention to letter processing and other attention-dependent measures related to functional reading. This paper reports on the methodology of a small feasibility study demonstrating proof of concept for this novel NFB training method for individuals with mild AD.

Materials and Methods

Participants

Six individuals with possible or probable AD were recruited as study participants. Five participants were enrolled; this was the target number for this feasibility study. One participant was not enrolled because of study visit demands. The participant number for this US National Institutes of Health-funded study was small but is typical for the planned analysis: a non-experimental multiple baseline single case research design (SCRD) [39,40]. Ultimately a conventional randomized controlled trial will be best for demonstrating efficacy of a NFB intervention, given concerns ranging from financial issues to placebo and expectancy effects [41,42]. However, the high cost of this particular NFB intervention, requiring many visits to participants’ homes or other preferred locations, makes the SCRD ideal for this small feasibility study. Due to safety concerns involved in conducting home visits for data collection with vulnerable populations during the COVID-19 pandemic, the study was concluded with complete data from only two participants. These participants, #1 and #2, will be referred to as “completers”. Data collection for the other three participants was discontinued before or shortly after the beginning of the intervention phase. Please refer to Table 1 for participant description and demographic information.

 

 

 

Participants were recruited from the Oregon Health & Science University (OHSU) Layton Aging and Alzheimer’s Center, which is funded in part as an NIH Alzheimer’s Disease Research Center. Recruitment procedures included chart review, communication with the treating neurologist regarding eligibility criteria for diagnosis of pAD, and a phone screening that consisted of approximately 30 minutes of questioning to determine eligibility. During this phone screening, participants completed the judgment subtest of the Neurobehavioral Cognitive Status Examination (NCSE) in order to assess capacity to consent to the study procedures [43]. Inclusion criteria were as follows: (1) adults aged 50-100 years old; (2) a diagnosis of mild possible or probable AD by a medical diagnosis by their treating neurologist at the Layton Aging and Alzheimer Disease Center consistent with conventional criteria [44] (3) a Global Clinical Dementia Rating (CDR) of 0.5 or 1.0 [45], (4) a Montreal Cognitive Assessment (MoCA) score of ≥ 14 [46] or a Mini Mental State Examination (MMSE) ≥ 18 [47] within the previous two months; (5) a mild language impairment attributable to cognitive challenges assessed through clinical impression by either an ASHA-certified speech-language pathologist from the research team (DEM) or behavioral neurologist at the Layton Center; (6) absence of EEGaltering medications (e.g. benzodiazepines); (7) no significant motor, vision, or hearing impairment; and (8) greater than 80% accuracy on at least one of four trials for a practice RSVP task (described below). This study was approved and overseen by the OHSU Institutional Review Board (IRB #18984) and registered with ClinicalTrials.gov (NCT03790774).

To ensure compliance with the study procedures and facilitate communication and scheduling of study activities, participants were required to enroll with a study partner who also consented to participation in the study. Both participants and study partners provided written informed consent. Study participants were paid at each visit, with a completion bonus paid after follow-up; study partners were not paid for their participation. In order to qualify as a study partner, an indivdidual needed to: (1) have a close relationship with the participant (e.g. spouse, adult child, or friend); (2) interact an average of at least 5 hours per week with the study participant, either in person or over the phone; (3) demonstrate capacity to consent as evidenced by a score of ≥3 on the NCSE Judgement subtest; and (4) demonstrate low risk of cognitive impairment, as measured by a score of ≥31 on the modified Telephone Interview for Cognitive Status (TICS-m) [48]. The study partners consisted of one brother, one son, one daughter, one mother, and one spouse.

Procedure

This study followed a within-subject A-B design with multiple data collection sessions both before and during administration of the intervention (hereafter “baseline” and “intervention”, respectively). In the baseline phase, participants received RSVP Keyboard training without NFB. In the intervention phase, the training incorporated NFB. A single follow-up session was conducted 4-5 weeks after the end of the intervention phase. All sessions were conducted at either the participant’s home, the Oken Laboratory at OHSU, or a neutral location, e.g., a public library meeting room, according to the participant’s preference. Consistent days of the week, start times, and visit durations were maintained for each participant. Please refer Figure 1 for an outline of study phases and the activities and assessments conducted at each visit; these are described below.

Figure 1: Study activities table. This infographic depicts the assessments conducted at each visit type and the frequency of their measure across phrases of the study. The dashed lines in the baseline phase represent the variability in number of baseline visits that varied between 4 to 7 visits based on stability of baseline performance from week-to-week.

Study entry visit and outcome measures

The first visit included informed consent procedures, study eligibility questions, and administration of initial summative outcome measures. The summative outcome measures, given once prior to the baseline sessions and once at the final follow-up session, were: (1) the Discourse Comprehension Test (DCT) [49], a measure of listening comprehension; and (2) the forward and backward digit span subtests of the Wechsler Adult Intelligence Scale IV (WAIS-IV), a measure of phonological working memory and attention [50]. The Discourse Comprehension Test requires a participant to listen to a 150 to 200 word short story and respond to comprehension questions [49]. There are five short stories per test form with eight questions about each story [49]. The digit span subtests in the WAIS-IV require a participant to listen to a sequence of numbers read aloud and repeat that sequence back to the examiner in the same order (forward) or in reverse order (backward) [50]. The number of digits in each sequence ranges from two to 16 [50]. Participants answered questions about their health, demographics, and the nature of their relationship with their study partner. Additionally, participants completed a computer-based practice RSVP Keyboard task without EEG set up or input; this was designed to familiarize them with the demands of the experimental task and to confirm that they could demonstrate the requisite skills, including: (1) attending to targets; (2) responding if a target was present on screen; and (3) inhibiting responses to non-targets. Participants were presented with a target letter and asked to attend to the letters in a series of 10 letters in order to answer the question: “Was the target letter in the sequence?” Five out of 10 sequences contained the target and five did not. Participants were trained from slowest presentation speed (1 Hz) to fastest presentation speed (4 Hz) in four steps (i.e., 1, 2, 3, 4 Hz). At each step, participants were trained to criterion, defined as getting eight out of 10 items correct at a given presentation rate (1 Hz, 2 Hz, 3 Hz, 4 Hz). Participants were provided up to four chances to train to criterion at a given presentation rate. An exclusion criteria of study enrollment was not achieving 80% accuracy at the 4 Hz presentation rate. No participants were excluded for this reason. This practice RSVP task was repeated at the beginning of each baseline, intervention, and follow up session in order to ensure that participants maintained an ability to complete the task with a presentation rate of 4 Hz.

Baseline visits and outcome measures

Baseline sessions were planned to begin one week after the initial visit, and to occur once per week afterwards. Participants completed four to seven baseline sessions, until there was reasonably stable performance of the outcome measures as assessed by visual analysis [51]. During each baseline session, participants completed RSVP Keyboard calibration and copyspelling tasks [52] using the BciPy software [38], as well as repeated measures outcome tasks to monitor progress. The repeated measures tasks were: (1) letter cancellation task; (2) letter span task; and (3) Woodcock Johnson Test of Achievement 4th edition (WJTA-IV) Sentence Reading Fluency Subtest (form A, B, or C) [53]. A description of each task follows. Participants were not excluded due to performance on these metrics.

For all RSVP Keyboard tasks (with exception to practice task), participants wore a dry electrode cap (Wearable Sensing; San Diego, CA) that measured EEG responses to target and nontarget letters (see section on Electrophysiological Recordings and Processing below for more details on EEG recording).

In the BCIpy calibration task, the participants were shown a single target letter and asked to attend to a rapidly-presented series of letters (nine non-target and one target), looking for the target letter. Each letter was presented centrally on the screen, one at a time, at a rate of 3 Hz. The temporal position of the target letter in each sequence was randomly assigned. For each of 100 sequences of letters, participants mentally responded when they saw the target on the screen. The intention of this paradigm was to elicit a P300 signal in response to target letter presentations, and to gather data for training a classifier to be used in the copy-spelling task. The calibration task lasted approximately 13 minutes. The main outcome measure in calibration was area under the curve (AUC), a measure of classification accuracy. Software and specifics related to the BciPy classifier are more fully outlined in Memmott and colleagues (2021). For classification, the software uses a regularized discriminant analysis with 10-fold cross validation to evaluate EEG in the 500 ms epoch following task stimuli.

In the copy-spelling task, participants were asked to copy a phrase letter by letter (“HELLO_” followed by a word of their choice) by selecting each target letter in the phrase from a stream of nontarget letters. Participants were instructed to mentally react to a target letter, as in the calibration task. For each sequence, the system combined EEG evidence with probabilities determined by an integrated language model [52] repeating sequences until one letter reached a probability threshold of 0.80 and was selected for typing. This task demands vigilance to the target for several sequences, as the target letter typically must be selected more than once before the system comes to a decision. The main outcome measure for the copy-spelling task were whether or not the target word was successfully copied.

The letter cancellation task in this study was adapted from the letter cancellation task used in Baddeley and colleagues (2001), and measured selective attention. Participants were instructed to cross out all instances of a target letter, “Z”, as quickly as possible, in a grid of upper-case letters presented on an 8.5 x 11 in. sheet of paper in 14-point Arial font [54]. There were two forms that varied in difficulty: an easier version with a combination of 10 curved letters as foils (e.g., “B", "P", and "R") and a harder version with a combination of 10 straight letters as foils (e.g., "K", "M", and "Y"). Baddeley and colleagues (2001) found that participants with AD took significantly longer on a version with straight letter foils, and proposed that this effect was attributable to the similarities of visual features between the non-target letters and the target letter “Z” [54]. There were 20 “Z” targets on each form out of a total of 150 letters. Target positions were randomized and plotted by assigning random x and y values to a 10 by 15 grid using R version 3.6.1 [55]. Randomly-generated form versions with more than three adjacent targets in a row, column, or corner were rejected. The outcome measure for this task was total completion time (in seconds) corrected for task accuracy (total time/accuracy), as used in other studies.

The letter span task in this study, which measures working memory, was adapted from the letter span task used by Conrad and Hull with task considerations modeled from the WAIS digit span subtest [50,56]. Participants attended to a sequence of two to eight letters, presented one at a time at a rate of one per second on a monitor, and recite the sequence back to the examiner either in the same order (forward condition) or in reverse order (backward condition). There were two items for each sequence length in the task. The task was discontinued when the participant answered both items for a given sequence length incorrectly. The letter span task was programmed in Python using PsychoPy3 v3.0.0b11 [57]. Strings consisted of only consonant letters to reduce the ability for participants to use a word encoding strategy. Sequences were reviewed by three researchers to remove any consonant combinations commonly used as acronyms or abbreviated phrases in English (e.g., BRB, HQ, RSVP). Stimulus order was randomized and 15 unique versions were generated to reduce the chance of a repetition learning effect across baseline, intervention, and follow-up weeks for each condition. Participants received unique versions in a randomized order. The outcome measure for this task was maximum sequence length, defined as the longest sequence length where a participant recited at least one of the two sequences correctly.

The well-validated WJTA-IV Sentence Reading Fluency subtest [53] was used to measure processing speed. This subtest requires participants to read as many sentences as possible and answer whether the sentence is generally “True” or “False” in three minutes. Example items include: “Fire is hot”, “Dogs can eat”, and “A school bus has a driver”. In order to minimize a repetition learning effect over baseline, intervention, and follow up sessions, the order of the three unique test forms was randomly permuted for each participant (e.g., C, A, B, repeating). The outcome measure for this task was the number of items answered correctly in a three-minute period.

Intervention visits and outcome measures

The decision to begin the intervention phase was determined by observation of reasonably stable baseline performance on the three repeated outcome tasks and the BCI calibration task as assessed by visual analysis [51], or by the participant reaching the pre-determined maximum of seven baseline sessions. Intervention sessions were completed three times per week for a six-week period, and required participant to complete a calibration task with NFB and a copy-spelling task. In this phase, the standard RSVP Keyboard calibration task from baseline was adjusted to feature an additional NFB display after each letter sequence. This display was onscreen for two seconds and included five colored boxes ranging from dark red (poor attentional performance) to dark green (excellent attentional performance; (Figure 2). A thick white border around one box was used to indicate to participants their attention rating on the most recent sequence. Participants were asked to try and achieve as many dark green (excellent) ratings as possible during each session and to pay more attention to the RSVP Keyboard sequences if they were given dark red, orange, yellow, or light green ratings. NFB was individualized for each participant and updated weekly based on data from the previous week’s calibration task (see Calculation of Neurofeedback below). To monitor progress, participants completed repeated measures tasks before the calibration and copy-spelling tasks during the third session of each week.

Figure 2: Schematic of the RSVP task and the neurofeedback to the participant. Following presentation of the target letter there is a sequence of 10 letters presented following a red cross-hair warning signal. Neurofeedback is based on individualized alpha Power Spectral Density (PSD) percentiles at a pre-specified occipitoparietal electrode. Colored boxes range from upper 15th percentile alpha power in red, 15th - 30th percentile, 30th - 45th, 45th to 70th, and the least alpha power in green from the 70th - 100th percentiles. The current feedback the participant sees is highlighted with the white edges around one colored rectangle.

Calculation of neurofeedback

In order to quantify attention to the RSVP Keyboard display for NFB, posterior parieto-occipital alpha power was used to measure engagement of visual attention. This signal was used for NFB because: (1) event-related alpha attenuation during visual tasks is associated with mental effort [58,59]; (2) alpha power has been used in prior NFB experiments during rest states to improve visual cognition [60]; and (3) results of a pilot study (below) supported its utility for this purpose.

In the alpha power pilot study, participants without known disabilities (n=8, age range 21-65 years) performed a one-back task that required them to press a button to indicate whether a target sequence was present within a longer sequence of ten letters. The target sequence was always the letter N followed by a random letter, followed by the letter A. Non-target sequences were included to increase difficulty, and consisted of the letters N and A separated by either two or zero other characters. While performing this task, participants wore a 24-channel wet electrode EEG system (BioSemi, Amsterdam). Average button-press error rate for the task was 10%, although three participants made fewer than two errors and were excluded from these analyses. For the five remaining participants, EEG frequency analysis of the 2.5-second epoch including each ten-letter sequence revealed a 17% increase in posterior rhythm (alpha) amplitude when participants made errors compared to when they made no errors (p=0.013, Figure 3). For the three participants who made no errors on the behavioral task, visual inspection of EEG data revealed that they had the lowest amplitude posterior alpha rhythm of the eight participants, further supporting the relationship between attention and posterior alpha rhythm.

Figure 3: EEG frequency data from 5 out of 8 participants who made errors during the alpha power pilot task. Spectra are from site Oz, averaged across 2.5 s epochs during sequences where participants either made or did not make recognition errors. Besides the increased alpha power during error trials, note the steady-state visual evoked potential aligned with presentation rate of 4 Hz, which did not change based on error status. The three participants not included in this average because they did not make errors had even lower levels of alpha than the “correct” condition illustrated in the figure.

Calibration recordings from all available baseline sessions were reviewed in order to determine which of five candidate electrode sites (Pz, Oz, P4, PO7, or PO8) recorded the greatest amount of resting alpha rhythm without interfering artifact for each participant. Due to poor contact at sites Pz and Oz and excessive electromyography (EMG) artifact at sites Oz, PO7, and PO8 for the first two participants, P4 was selected as the channel of interest for calculating NFB during intervention for the completers, participants #1 and #2. Pz was utilized for NFB for participant #4, who finished six intervention sessions over two weeks.

Both participants who completed the study visits consistently demonstrated peak resting alpha activity at approximately 9.0 Hz in both baseline and intervention sessions, therefore, a target frequency band of 8-10 Hz was designated for NFB. All EEG feedback data were bandpass filtered at 7-20 Hz in order to minimize interference of both EMG and a 6 Hz harmonic related to the 3 Hz steady-state visually evoked potential elicited by the RSVP Keyboard letter stream. Relative power spectral density (PSD; μV2/Hz) was calculated using BciPy’s signal decomposition module with Welch’s method [61] and defined as the PSD of the target band (8-10 Hz) compared to PSD of the wider band (7-20 Hz). From aggregate baseline data within-participants, the 70th, 45th, 30th, and 15th percentiles of relative PSD were set as the NFB cutoffs for the first week of intervention. Specifically, the delineation of the lowest 30% of relative alpha amplitude responses (70th percentile) was used to demarcate dark green “excellent” feedback; the next lowest 25% of relative alpha PSD values (45th percentile cutoff) marked the range of light green “good” feedback, and the remaining 45% of relative PSD values were equally divided into orange (“medium”), yellow (“poor”), and dark red (“bad”) ranges, respectively (Figure 2). These thresholds were chosen in order to provide a positive bias to the feedback and encourage participant engagement. The feedback was presented after each ten-letter sequence (with letters shown at three per second), so approximately every three seconds. After each week of intervention, these cutoffs were recalculated from the most recent week of calibration data.

Follow up visit

Approximately one month after the final intervention session, participants completed one follow-up session to assess maintenance of performance on the repeated measures tasks, as well as a final RSVP Keyboard calibration and copy-spelling session without NFB.

Electrophysiological recordings and processing

EEG data were recorded using an adjustable DSI VR300 dryelectrode system (Wearable Sensing; San Diego, CA) with linkedear references, ground at A1 (left earlobe), and scalp electrodes at 10/20 sites FCz, F7, Pz, P4, PO7, PO8, and Oz. Data were sampled at 300 Hz and digitized at 16 bits. The inclusion of an electrode at F7 was a modification of the standard VR300 design (the original location was P3), made to allow monitoring of electrooculogram. Though field recordings of EEG are sometimes more susceptible to noise than data collected in a more controlled laboratory setting (e.g. ambient 60 Hz electrical noise from power sources in US; mechanical artifact related to appliances or home-medical equipment etc.), all electrodes were calibrated to be within operating ranges as specified by the manufacturer guidelines. Additionally, experimenters monitored real-time EEG during the tasks (displayed using 2-45 Hz bandpass and a 60 Hz notch filters) in order to identify and minimize high amplitude artifacts and ambient noise.

All recordings were down-sampled to 150 Hz and filtered 2-45 Hz for analysis by the classifier. N200 and P300 potentials at site Pz were later quantified offline in Brain Vision Analyzer (Brain Vision LLC; Morrisville, NC, U.S.A.) in order to explore target discrimination and identification processes, respectively. Offline data were band pass filtered 1-45 Hz with a 60-Hz notch before use of independent component analysis (ICA) to remove eye blinks. These data were segmented -200ms to 1000ms relative to target and non-target letters before baseline correction using the 200 ms prior to letter presentation and then subjected to artifact rejection. Epochs were flagged for review if they contained voltage steps >50 μV/ms, amplitude changes >125 μV over 50 ms, amplitude values > ± 75 μV, or sustained amplitude values < 0.5 μV for longer than 100ms. N200, and P300 peaks were cursored using semiautomatic peak detection windows of 250-400, and 350-500 ms, respectively. N200 and P300 peaks were quantified for analysis using peak-to-trough, or by measuring the change in voltage from the most recent peak of the opposite polarity.

Data analysis

One analysis component for this non-experimental single case research design, as is typical for all SCRD studies, was graphical analysis [40]. The graphical analysis is particularly useful for heterogenous populations allowing for iterative improvements.

However, the multiple repeated baseline and intervention-phase measurements made possible conventional longitudinal statistical analyses as well. We performed a variety of within-subject and between-subject analyses in order to gain better insight into the reliability and informativeness of the outcome measures in the study population.

For analysis of the outcome measures, we omitted the first baseline measurement for each participant on each task. In nearly every case the first recorded outcome measurement was an atypical value, either sharply higher or lower than the next measurement. We interpret this as an acclimation effect of no significance for the study. The first recorded values for outcomes are shown on plots for completeness. For baseline stability and correlation analyses, all baseline points were included.

To evaluate the stability of the measures of interest, we calculated within-subject coefficients of variation (CVs) and intraclass correlation coefficients (ICCs) using the baseline measurements only. The CV is a metric of overall volatility in a longitudinal setting, expressing the equilibrium standard deviation as a fraction of the longrun mean for the participant. Lower CVs are better, and, as a rule of thumb, values of ~5% or lower are indicative of good stability and values in excess of 10% are indicative of poor stability. We estimated an average within-subject standard deviation as the root-mean-square error from an absorbing regression [62] on the baseline measurements of the cohort (absorbing participant effects), and calculated the average within-subject CV as this value divided by the overall baseline mean. (Note that this is a population-level estimate, not a strict average of the individual CVs.) The standard error of the CV was approximated using the formula

where , for n=5 (the number of participants), m=4.4 (the average number of included baseline measurements per participant), and r (an estimate of the average correlation among longitudinal values for a participant) different for each outcome. ICCs and their corresponding confidence intervals were estimated using a restricted-maximum-likelihood linear mixedeffects model [63] of the outcome measure adjusted for session time, specifying random intercepts for participants. The ICC is an estimate of the average proportion of total variance attributable to true differences in the outcome measure.

Within- and between-subject correlations of median relative alpha power and other EEG-derived metrics (e.g. average P300 amplitude across all target letter events) were calculated with a fixed-effects longitudinal model [64] using Bland and Altman’s method [65] for the within-subject correlation estimate and the complementary "between-effects" longitudinal estimator (used in the calculation of the fixed-effects model) for the betweensubject estimate. Degrees of freedom for the within-subject correlation were estimated using the df formula noted above, and using the sample size n=5 for the between-subject correlation, in each case subtracting 3 when calculating Fisher's approximation of the z-score of the correlation [66].

Longitudinal slopes over the baseline and intervention phases for each outcome measure were estimated using an ordinary linear regression of the outcome on time (in weeks), study phase (0=baseline and 1=intervention), and the time-by-phase interaction, fitted separately for each participant on each outcome. In order to account for influence of the longitudinal correlation in measured values on the measurement variance, we employed the Newey-West robust variance estimator [67] for calculating the standard errors for the slopes. Within-phase means and across-phase changes were also calculated from this model using Newey-West standard errors. To visualize the trajectory and evaluate the intervention effect on each outcome for each participant, we plotted the measurements of the outcome versus time and overlaid slopes for the baseline phase (omitting the first point) and the intervention phase. For all outcome measures, we omitted the final point from slope calculations; this point represents the follow-up session, which occurred approximately one month after the end of the intervention phase.

Stata version 16.1 [68] was used for the statistical analyses listed above, and R version 3.6.1 [55] was used for data management and to generate descriptive summaries.

Results

Five participants with mild pAD enrolled in the study (Refer Table 1 for demographic information,). All five completed the baseline phase, and two completed all baseline, intervention, and followup sessions. All repeated measure results for each participant are included in Table 2.

Participant	Age	Years of Education	MoCA	#Baseline Sessions	#Intervention Sessions
#1	79	16	21	4	18
#2	66	17	29	5	18
#3	53	18	22	4	-
#4	72	18	24	7	6
#5	76	13	19	7	-

Table 1: Participant demographics. All participants were white and non-Hispanic.

Participant	#of Sessions	Measure	Baseline	Intervention
			(M ± SD)	(M  ±  SD)
#1	Baseline: 4 Intervention: 18	Letter Cancellation
		Curved Letters	52.03 ± 5.42	46.56 ±3.55
		Straight Letters	54.19 ± 2.65	51.51 ± 5.53
		Letter Span
		Forward	4.25  ± 0.5	4 ± 1.26
		Backward	3 ± 0	2.67 ± 0.52
		WJTA-IV SRF	54 ± 4.9	55 ± 5.55
#2	Baseline: 5 Intervention: 18	Letter Cancellation
		Curved Letters	31.76 ± 2.74	29.35 ± 2.07
		Straight Letters	47.5 ± 5.79	37.93 ± 2.38
		Letter Span
		Forward	4 ± 1	4.6  ± 0.81
		Backward	3.4 ± 0.89	4  ± 0
		WJTA-IV SRF	77.2 ± 7.08	89.5 ± 5.89
#3	Baseline: 4 Intervention: 0	Letter Cancellation
		Curved Letters	47.55 ± 16.90	--
		Straight Letters	57.21 ± 14.24	--
		Letter Span
		Forward	5.25  ± 0.5	--
		Backward	4 ±  0	--
		WJTA-IV SRF	61.25 ± 8.18*	--
#4	Baseline: 7 Intervention: 6	Letter Cancellation
		Curved Letters	35.8 ± 2.48	33.9 ± 1.53
		Straight Letters	40.01 ± 3.31	39.88 ± 3.53
		Letter Span
		Forward	4.86 ± 0.9	6  ± 0
		Backward	4.43 ± 0.53	5  ± 0
		WJTA-IV SRF	49.43 ± 3.26*	48 ± 7.07*
#5	Baseline:  7 Intervention: 0	Letter Cancellation
		Curved Letters	39.67 ± 2.35	--
		Straight Letters	50.44 ± 4.54	--
		Letter Span
		Forward	3.71 ± 0.49	--
		Backward	2.85 ± 0.38	--
		WJTA-IV  SRF	56.2 ± 3.27	--

The number of sessions is included for reference as only two participants (participant #1 and #2) completed the study in its entirety. The asterisk (*) is used to demonstrate which participants performed below expectations as defined as deviation from the expected raw score based on comparison sample of same-aged peers without known impairments. For the letter cancellation task, no measure of variance or range of performance was included in the descriptive statistics for any participant group. It is therefore not reliable to make a judgement on participants’ performance without a range of performance. Averages are included as a reference of performance. Same-aged peers without AD completed the letter cancellation task in an average of 25 seconds for curved letters and 32 seconds for straight letters (Baddeley et al., 2001). Participants with AD completed the letter cancellation task in an average of 35 seconds for curved letters and 49 seconds for straight letters (Baddeley et al., 2001). For the WJTA-IV sentence reading fluency subtest, average performance as defined as receiving a standard score between 90 and 110 is equivalent to a raw score of 58-85 for ages 50-59, 54-80 for ages 60-69, and 49-74 for ages 70-79 (normative sample used for comparison). There was no same-aged peer group in literature for letter span task to make comparisons of performance.

Table 2: This table shows the raw score performance on all repeated measures for both baseline and intervention phases for each participant.

Baseline stability of outcome measures

The stability of the outcome measures across baseline sessions is shown in Table 3. In particular, performance on WJTA-IV Sentence Reading Fluency, letter cancellation with curved foils, and AUC was reasonably stable, with CVs less than 10%. Some measures demonstrated a learning effect within the baseline phase for some participants, which can be seen in both the AUC results shown in Figure 4, outcome measure results for participants #1 and #2 (completers) in Figure 5.

Measures	CV ± SE (%)	ICC [95% CI]
AUC	9.23 ± 2.57	0.68 [0.28,0.92]
Median Relative Alpha PSD	24.90 ± 6.68	≈0
WJTA-IV: Sentence Reading Fluency Raw Score	6.50 ± 1.98	0.90 [0.65,0.98]
Maximum Letter Span: Forward	19.01 ± 4.11	0.22 [0.02,0.80]
Maximum Letter Span: Backward	16.04 ± 4.20	0.57 [0.18,0.89]
Letter Cancellation (sec): Curved	5.19 ± 1.59	0.92 [0.71,0.98]
Letter Cancellation (sec): Straight	13.38 ± 3.17	0.37 [0.06,0.83]

Table 3: Within-person baseline outcome measure stability (excluding the first baseline session for each participant), averaged across the five participants. Coefficients of Variation (CV) greater than 10% indicate poor stability. Given the number of participants, the wide range of ICC estimates is difficult to interpret but consistent with the CVs. ICCs are very high for WJ-Sentence Reading Fluency and Letter Cancellation Time: curved-letter foils.

Figure 4: AUC scores for five participants. BCI sessions were conducted once per week in the baseline phase, and three times per week in the intervention phase. Curves showing the trajectory of AUC for each participant across the study duration were calculated using a running-mean lowers smoother with tricube weighting and default bandwidth. Three participants demonstrated a learning effect during the baseline. One participant (#1) continued to improve during the intervention, with AUC values rising to 0.90. Another participant with a particularly low AUC (#4), in some sessions about 0.5, had no discernible N200-P300 in the averaged ERP waveforms.

Figure 5: Full data from repeated measures tasks administered in baseline sessions, intervention sessions, and follow-up session for the two participants who completed the study. In general, the WJTA IV Sentence Reading Fluency, Letter Cancellation with curved foils, and AUC (Figure 2) were the most reliable. Note examples of outcome measures in for which there was: no change in the intervention (Letter Span Backward) (d and i); a sustained learning effect continuing through the baseline and then through the intervention periods (Sentence Reading Fluency) (j); a learning effect only during the baseline (Letter cancellation with straight foils) (b and g); an apparent improvement from the intervention (Letter Cancellation with curved foils) (a), and a decrease in performance in treatment phase (Letter Span forward) (c).

On the RSVP Keyboard calibration task, the five participants achieved a mean correct classification rate (AUC) across their baseline sessions of 0.72 ± 0.03 (mean ± SE). In the baseline phase, AUC for participants #1, #2, and #3 ranged from 0.67 to 0.80, 0.69 to 0.83, and 0.58 to 0.76, respectively. Participants #4 and #5 (both non-completers) exhibited consistently low AUC values near 0.6 across their baseline sessions (Figure 4). Illustrations of representative EEGs and ERPs taken from the 1st week of intervention for participants #1 and #2 (completers) and participant #4 (non-completer) are presented in Figure 6.

Figure 6: (A) 5-seconds of representative EEG data taken from week #1 of intervention. The presented windows each include one full sequence of 10 letters (1 target; 9 non-targets) during the RSVP neurofeedback calibration task. Parieto-occipital alpha is clearly visible at sites P4 and Pz prior to fixation. Participant #1 demonstrates neck EMG contamination at posterior sites Oz, PO7, and PO8. (B) Demonstrative ERP averages of target and non-target responses at Pz, derived from week #1 of intervention (3 sessions; neurofeedback calibration). Participant #1 shows EMG contamination, but a large N2/P3 response. Participant #2 exhibits alpha signal but also a small yet clear ERP response to the target. Participant #4 demonstrates no visible target-related response.

In the outcome measures tasks, letter cancellation tasks for both participants (Figures 5A, 5B, 5F and 4G) and WJTA-IV Sentence Reading Fluency for participant #2 (Figure 5J) demonstrated baseline variability.

Effects of intervention

Most outcome measures that were repeated multiple times before and during the intervention did not change significantly across phases (Table 2, Table 4, and Figure 5). Figure 5 includes examples for which there was no change in performance with the introduction of the intervention (Figure 5D), a sustained learning effect continuing through the baseline and intervention phases (Figure 5J), a learning effect only during the baseline phase (Figure 5G), and an apparent improvement from the intervention (Figure 5A). Participant 1 demonstrated a decrease in performance in the intervention phase for the letter span forward condition but not the letter span backward condition. As the letter span backward condition is of higher complexity and may be a better index of working memory, it is believed that motivational factors or fatigue related to the difficulty of the task may have influenced this participant’s performance.

Measures	Slope ± SE	Participant
		#1	#2
SRF Raw Score	Baseline	0.0 ± 4.1	2.7 ± 2.2
	Intervention	0.9 ± 1.4	1.9 ± 1.2
Max Letter Span: Forward	Baseline	0.0 ± 0.4	-0.8 ± 0.3
	Intervention	-0.6 ± 0.1	0.3 ± 0.2
Max Letter Span: Backward	Baseline	0.0 ± 0.4	0.6 ± 0.2
	Intervention	-0.1 ± 0.1	0.0 ± 0.1
Letter Cancellation (sec): Curved	Baseline	-0.4 ± 1.8	-1.1 ± 0.5
	Intervention	-1.5 ± 0.6	-0.9 ± 0.3
Letter Cancellation (sec): Straight	Baseline	-3.2 ± 3.8	-3.7 ± 1.6
	Intervention	-0.6 ± 1.3	-0.3 ± 0.9

Table 4: Longitudinal slopes over the baseline and intervention periods for each outcome measure. Slope estimates were generated using an ordinary linear regression of the outcome on time, and standard errors employed the Newey-West robust variance estimator. Note that a negative sign indicates improvement for some measures (both letter cancellation tests), but otherwise a positive sign represents improvement.

Figure 4 depicts AUC results by session for each participant. AUC changed by 0.14 ± 0.03 (mean ± SE) across the baseline (0.74 ± 0.02) and intervention (0.89 ± 0.01) phases for participant #1. Participant #2’s performance during the intervention phase was variable, with AUC ranging from 0.55 to 0.85 (0.74 ± 0.02).

In the copy-spelling task during the intervention phase, participant #1 (mean intervention phase AUC of 0.89) correctly copied the word “HELLO” in 15/17 opportunities, compared to 0/4 opportunities in the baseline phase. Participant #2 (mean intervention phase AUC of 0.74) copied the entire word in 3/5 opportunities in the baseline phase and 7/18 opportunities in the intervention phase, and typed “HELL" correctly in an additional 5/18 intervention phase opportunities but did not complete the word. The other three participants who did not undergo the intervention had lower AUCs (as low as 0.5) for some or all of their baseline sessions and thus, not unexpectedly, were usually unable to type even the “H” correctly during baseline sessions.

The summative outcome measures (WAIS Digit Span and Discourse Comprehension Test) were given once prior to and once after the intervention. There was no significant difference between these metrics pre- and post-intervention.

Neurofeedback, posterior alpha levels, and supplementary ERPs

The posterior alpha power NFB was successfully implemented. The five levels of feedback were delivered with the approximate planned target frequencies. Against a targeted best-to-worst distribution of 30%, 25%, 15%, 15%, and 15%, participant #1’s NFB demonstrated averages of 36.6%, 24.7%, 13.7%, 10.9%, and 14.1% across all intervention sessions. Participant #2’s NFB demonstrated averages of 31.3%, 23.1%, 16.3%, 14.6%, and 14.7%. There was no clear change over time in the posterior alpha power in the two completers. There was a significant correlation between target P300 amplitude and AUC across participants (0.94; z=2.46), but the within-subject relationship was not as strong (0.45; z=1.47). Analogously, there was a strong correlation between N200 amplitude and AUC between participants (-0.80; z=-1.57) and the within-subject correlation was absent (-0.04; z=-0.12). Representative samples of EEG and ERP responses to targets and non-targets are presented in Figure 6A in order to illustrate resting EEG including alpha activity, and typical ERP responses observed during the neurofeedback calibration task.

Discussion

In this study, two participants with mild pAD completed an intensive neurofeedback-based intervention study, with weekly home visits for over a month in the baseline phase and three visits per week for six weeks during the intervention phase. Three of five participants successfully took part in 3-5 baseline visits but were discontinued prior to or at the beginning of the intervention phase because of public health guidelines during the COVID-19 pandemic.

This study demonstrated that adults with mild pAD were able to perform the BCI RSVP Keyboard calibration (letter detection) task, with one participant achieving AUC values up to 0.94 but another of the five demonstrating AUC values as low as 0.5. The participant with very low AUC appeared engaged and was able to perform the letter discrimination task, but had essentially no P300 (Figure 6B, participant #4). There is some concern that this is at least partly related to aging that produces a decline in P300 amplitude as well as prolongation of P300 latency, and AD further increases those changes [69,70]. The task required participants to attend to 100 sequences of letters and look for a target while minimizing distractions to non-targets and artifact-inducing movements. This relies heavily on attention, a domain of cognition that is affected in early stages of AD. Additionally, participants who completed the intervention were able to correctly type words in a copy spelling task. This contributes to the BCI field by demonstrating that adults with impairments in attention, a domain of cognition, can learn to operate a BCI for communication.

Results for all five participants demonstrated that some, but not all, outcome measures were stable during the baseline phase. For instance, the WJTA-IV Sentence Reading Fluency, letter cancellation with curved foils, and AUC had CVs less than 10%. The novel letter span task with scoring similar to the WAIS Digit Span had worse baseline stability with higher CVs (19% for forward and 16% for backward). This demonstrates the stability of some measures for future studies that might be more robust to test-retest effects.

The current study demonstrated feasibility of a NFB intervention using posterior alpha power during performance of an RSVP Keyboard task. Some outcome measures showed no effect from the NFB intervention (e.g. letter span backward), and some demonstrated either a learning effect during the baseline (e.g. letter cancellation with straight foils) or a continued learning effect across the baseline and intervention periods (e.g. WJTA-IV Sentence Reading Fluency). There was an apparent intervention effect in at least one measure, with improvement on the letter cancellation task with curved foils after NFB implementation, in one of the two completers. There was no consistent change in the posterior alpha power from the NFB intervention in the two completers. Although there were no significant changes in performance across phases on most measures, it is a notable finding to see a maintenance of performance on cognitive outcome measures in the context of Alzheimer’s disease, a degenerative condition impacting cognition.

This proof-of-concept study had limitations. The planned single case design was a non-blinded study with few participants. Both limit the conclusions as well as the ability to report many of the items that should be ideally reported in NFB studies, e.g., details of control group and blinding [71]. Individuals with pAD who demonstrate specific language or attention impairments related to the intervention targets may benefit more from the proposed intervention than those with cognitive impairments in domains that extend beyond target intervention areas (e.g. memory, visuospatial etc.). In addition, adults with stable neurological impairments, such as chronic aphasia, may improve performance more with this innovative intervention. EEG recordings obtained in a natural environment, such as the participant’s home, may be more susceptible to artifacts. However, as the purpose of this study was to assess the feasibility of an intervention not requiring participants to drive, and so it was important to conduct recordings in participants’ homes in order to facilitate participant compliance, limit attrition, and to observe real-life contributors to the efficacy of intervention (e.g. typical environmental noise, dry electrode system fitting challenges, discomfort, differences in alertness etc.). As with other behavioral research, future NFB studies will need to control for non-specific aspects of improvement that may have occurred with the additional social interaction and cognitive stimulation associated with data collection visits, independent of the NFB. It is possible that the baseline version of the RSVP Keyboard tasks provided stimulation that could be considered a potential intervention. The specific target for the NFB (alpha level) was not optimal for the study as; the utility of the NFB would have been supported by demonstrating changes in the EEG measure used for the NFB. Finally, the sample size was small, and homogenous in socioeconomic status, race, and education. Larger studies with control conditions and more diverse participant samples are needed to explore this question further.

Conclusion

In conclusion, these findings constitute novel contributions to the field of AD and BCI, even though this study did not demonstrate definitive improvements in outcome measures. First, the results of this study demonstrated the feasibility of this mode of intervention in participants with mild pAD. While this project required significant programming and signal processing expertise, the use of open-source software and the availability of clinical NFB devices dramatically increase the feasibility of further research and eventual clinical application. Single case design studies with multiple assessments before and during an intervention is a potential approach for AD pilot studies. Most importantly, we have clearly demonstrated that participants with cognitive impairments such as AD can use a P300 based BCI speller for calibration and two participants were able to use the RSVP P300 speller BCI system to spell a short word (“hello”) in an experimental setting. Additional research is necessary to investigate the extent and generalizability of these findings.

Acknowledgement

We would like to acknowledge Andy Fish and Mayling Dixon for their administrative contributions to the project. The Consortium for Accessible Multimodal Brain-Body Interfaces (CAMBI) was comprised of the following members during the time of this study (excluding authors who are affiliated with CAMBI): Deniz Erdogmus, David Smith, Steven Bedrick, Brandon Eddy, Michelle Kinsella, Matthew Lawhead, Aziz Kocanaogullari, and Shiran Dudy.

Funding

This research was financially supported by the grant sources R01 DC009834, DC009834-09S1, and the OHSU ADRC P30 AG066518. The sponsors had no role in study design, data collection, analysis, interpretation of data, writing of the paper, or decision for submission to publication.

36329

References

1. Storandt M, Stone K, LaBarge E (1995) Deficits in reading performance in very mild dementia of the Alzheimer type. Neuropsychology 9: 174.
2. Klimova B, Maresova P, Valis M, Hort J, Kuca K (2015) Alzheimer’s disease and language impairments: social intervention and medical treatment. Clin Interv Aging 10: 1401.
3. Kendeou P, Van Den Broek P, Helder A, Karlsson J (2014) A cognitive view of reading comprehension: Implications for reading difficulties. Learn Disabil Res Pract 29: 10-16.
4. Perry RJ, Hodges JR (1999) Attention and executive deficits in Alzheimer's disease: A critical review. Brain 122: 383-404.
5. Ball K, Berch DB, Helmers KF, Jobe JB, Leveck MD, et al. (2002) Effects of cognitive training interventions with older adults: a randomized controlled trial. JAMA 288: 2271-2281.
6. Willis SL, Tennstedt SL, Marsiske M, Ball K, Elias J, et al. (2006) Long-term effects of cognitive training on everyday functional outcomes in older adults. JAMA 296: 2805-2814.
7. Rolle CE, Anguera JA, Skinner SN, Voytek B, Gazzaley A (2017) Enhancing spatial attention and working memory in younger and older adults. J Cogn Neurosci 29: 1483-1497.
8. Sala G, Tatlidil KS, Gobet F (2018) Video game training does not enhance cognitive ability: A comprehensive meta-analytic investigation. Psychol Bull 144: 111.
9. Kallio EL, Öhman H, Kautiainen H, Hietanen M, Pitkälä K (2017) Cognitive training interventions for patients with Alzheimer’s disease: a systematic review. J Alzheimers Dis 56: 1349-1372.
10. Carrion C, Folkvord F, Anastasiadou D, Aymerich M (2018) Cognitive therapy for dementia patients: a systematic review. Dement Geriatr Cogn Disord 46: 1-26.
11. Barnes DE, Yaffe K, Belfor N, Jagust WJ, DeCarli C, et al. (2009) Computer-based cognitive training for mild cognitive impairment: results from a pilot randomized, controlled trial. Alzheimer Dis Assoc Disord 23: 205.
12. Anguera JA, Boccanfuso J, Rintoul JL, Al-Hashimi O, Faraji F, et al. (2013) Video game training enhances cognitive control in older adults. Nature 501: 97-101.
13. Toril P, Reales JM, Ballesteros S (2014) Video game training enhances cognition of older adults: a meta-analytic study. Psychol Aging 29: 706.
14. Clerc M (2013) Brain Computer Interfaces, Principles and Practise. Biomed Eng OnLine 12: 22.
15. Kubler A, Mushahwar VK, Hochberg LR, Donoghue JP (2006) BCI meeting 2005-workshop on clinical issues and applications. IEEE Trans Neural Syst Rehabil Eng 14: 131-134.
16. Akcakaya M, Peters B, Moghadamfalahi M, Mooney AR, Orhan U, et al. (2013) Noninvasive brain–computer interfaces for augmentative and alternative communication. IEEE Rev Biomed Eng 7: 31-49.
17. Rezeika A, Benda M, Stawicki P, Gembler F, Saboor A, et al. (2018) Brain–computer interface spellers: A review. Brain Sci 8: 57.
18. Evans JM, Dellinger M, Russell H (2020) Neurofeedback: The first 50 years, Academic Press, San Diego, CA, USA.
19. Sitaram R, Ros T, Stoeckel L, Haller S, Scharnowski F, et al. (2017) Closed-loop brain training: the science of neurofeedback. Nat Rev Neurosci 18: 86-100.
20. Paluch K, Jurewicz K, Rogala J, Krauz R, Szczypińska M, et al. (2017) Beware: Recruitment of muscle activity by the EEG-neurofeedback trainings of high frequencies. Front Hum Neurosci 11: 119.
21. Oken B, Memmott T, Eddy B, Wiedrick J, Fried-Oken M (2018) Vigilance state fluctuations and performance using brain–computer interface for communication. Brain-Computer Interfaces 5: 146-156.
22. Huang J, Hensch T, Ulke C, Sander C, Spada J, et al. (2017) Evoked potentials and behavioral performance during different states of brain arousal. BMC Neurosci 18: 1.
23. Arvaneh M, Ward TE, Robertson IH (2015) Effects of feedback latency on p300-based brain-computer interface. In2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2015: 2315-2318.
24. Arvaneh M, Robertson IH, Ward TE (2019) A p300-based brain-computer interface for improving attention. Front Hum Neurosci 12: 524.
25. Jirayucharoensak S, Israsena P, Pan-Ngum S, Hemrungrojn S, Maes M (2019) A game-based neurofeedback training system to enhance cognitive performance in healthy elderly subjects and in patients with amnestic mild cognitive impairment. Clin Interv Aging 14: 347.
26. Lavy Y, Dwolatzky T, Kaplan Z, Guez J, Todder D (2019) Neurofeedback improves memory and peak alpha frequency in individuals with mild cognitive impairment. Appl Psychophysiol Biofeedback 44: 41-49.
27. Ordikhani-Seyedlar M, Lebedev MA, Sorensen HB, Puthusserypady S (2016) Neurofeedback therapy for enhancing visual attention: state-of-the-art and challenges. Front Neurosci 10: 352.
28. Lim CG, Lee TS, Guan C, Fung DS, Zhao Y, et al. (2012) A brain-computer interface based attention training program for treating attention deficit hyperactivity disorder. PloS one 7: e46692.
29. Laborda-Sánchez F, Cansino S (2021) The Effects of Neurofeedback on Aging-Associated Cognitive Decline: A Systematic Review. Appl Psychophysiol Biofeedback 2021: 1.
30. Jiang Y, Abiri R, Zhao X (2017) Tuning up the old brain with new tricks: attention training via neurofeedback. Front Aging Neurosci 9: 52.
31. Lee TS, Quek SY, Goh SJ, Phillips R, Guan C, et al. (2015) A pilot randomized controlled trial using EEG-based brain–computer interface training for a Chinese-speaking group of healthy elderly. Clin Interv Aging 10: 217.
32. Yeo SN, Lee TS, Sng WT, Heo MQ, Bautista D, et al. (2018) Effectiveness of a personalized brain-computer interface system for cognitive training in healthy elderly: A randomized controlled trial. J Alzheimers Dis 66: 127-138.
33. Fotuhi M, Lubinski B, Trullinger M, Hausterman N, Riloff T, et al. (2016) A personalized 12‐week" Brain Fitness Program" for improving cognitive function and increasing the volume of hippocampus in elderly with mild cognitive impairement. J Prev Alzheimers Dis 3: 133-137.
34. Luijmes RE, Pouwels S, Boonman J (2016) The effectiveness of neurofeedback on cognitive functioning in patients with Alzheimer's disease: Preliminary results. Clin Neurophysiol 46: 179-187.
35. Jang JH, Kim J, Park G, Kim H, Jung ES, et al. (2019) Beta wave enhancement neurofeedback improves cognitive functions in patients with mild cognitive impairment: A preliminary pilot study. Medicine 98: e18357.
36. Surmeli T, Eralp E, Mustafazade I, Kos H, Özer GE, et al. (2016) Quantitative EEG neurometric analysis–guided neurofeedback treatment in dementia: 20 cases. How neurometric analysis is important for the treatment of dementia and as a biomarker?. Clin EEG Neurosci 47: 118-133.
37. Lucas Koberda J (2014) Z-score LORETA neurofeedback as a potential therapy in cognitive dysfunction and dementia. J Psychol Clin Psychiatry 1: 00037.
38. Memmott T, Koçanaoğulları A, Lawhead M, Klee D, Dudy S, et al. BciPy: brain–computer interface software in Python. Brain-Computer Interfaces 2021: 1-8.
39. Gabler NB, Duan N, Vohra S, Kravitz RL (2011) N-of-1 trials in the medical literature: a systematic review. Med Care 1: 761-768.
40. Ledford JR, Gast DL, editors (2014) Single case research methodology: Applications in special education and behavioral sciences. Routledge.
41. Thibault RT, MacPherson A, Lifshitz M, Roth RR, Raz A (2018) Neuro feedback with FMRI: A critical systematic review. NeuroImage 172: 786-807.
42. Oken BS (2008) Placebo effects: clinical aspects and neurobiology. Brain 131: 2812-2823.
43. Kiernan RJ, Mueller J, Langston JW, Van Dyke C (1987) The Neurobehavioral Cognitive Status Examination: a brief but quantitative approach to cognitive assessment. Ann Intern Med 107: 481-485.
44. McKhann G, Drachman D, Folstein M, Katzman R, Price D, et al. (1984) Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA work group under the auspices of department of health and human services task force on Alzheimer ’s disease. Neurology 34: 939-944.
45. Morris JC (1993) The Clinical Dementia Rating (CDR): Current version and scoring rules. Neurology 43: 2412.
46. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, et al. (2005) The Montreal Cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53: 695-699.
47. Upton J (2013) Mini-Mental State Examination. In Encyclopedia of Behavioral Medicine, New York, NY. Springer; pp: 1248-1249.
48. Knopman DS, Roberts RO, Geda YE, Pankratz VS, Christianson TJ, et al. (2010) Validation of the telephone interview for cognitive status-modified in subjects with normal cognition, mild cognitive impairment, or dementia. Neuroepidemiology 34: 34-42.
49. Brookshire R, Nicholas L (1993) Discourse Comprehension Test: The Psychological Corporation, Tucson, AZ.
50. Wechsler D (2008) Wechsler Adult Intelligence Scale, Pearson, San Antonio, TX, USA.
51. Gast DL, Spriggs AD (2014) Single case research methodology: Applications in special education and behavioral sciences. Visual Analysis of Graphic Data Routledge, USA. pp: 176-210.
52. Oken BS, Orhan U, Roark B, Erdogmus D, Fowler A, et al. (2014) Brain-computer interface with language model–electroencephalography fusion for locked-in syndrome. Neurorehabil Neural Repair 28: 387-394.
53. Mather N, Wendling BJ (2015) Essentials of WJ IV® tests of achievement, John Wiley & Sons Inc, Hoboken, NJ, USA.
54. Baddeley AD, Baddeley HA, Bucks RS, Wilcock GK (2001) Attentional control in Alzheimer’s disease. Brain J Neurol 124: 1492-1508.
55. R core team (2013) R: A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, Austria.
56. Conrad R, Hull AJ (1964) Information, Acoustic Confusion and Memory Span. Br J Psychol 55: 429-432.
57. Peirce J, Gray JR, Simpson S, MacAskill M, Höchenberger R, et al. (2019) PsychoPy2: Experiments in behavior made easy. Behav Res Methods 51: 195-203.
58. Pfurtscheller G, Lopes FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110: 1842-1857.
59. Thut G, Nietzel A, Brandt SA, Pascual-Leone A (2006) α-Band Electroencephalographic Activity over Occipital Cortex Indexes Visuospatial Attention Bias and Predicts Visual Target Detection. J Neurosci 26: 9494-9502.
60. Schwartz MS, Andrasik F, eds. (2016) Biofeedback: A pracitioner’s guide, The Guilford Press, New York, NY, USA.
61. Welch P (1967) The use of the fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms. IEEE Trans Audio Electroacoust 15: 70-73.
62. Blackwell JL (2005) Estimation and Testing of Fixed-effect Panel-data Systems: Stata J 5: 202-207.
63. Diggle P, Liang K, Zeger S, Heagerty P (2002) Analysis of Longitudinal Data, Oxford University Press, New York, USA.
64. Allison P (2009) Fixed Effects Regression Models, SAGE Publications, Inc., 2455 Teller Road, Thousand Oaks California 91320, USA.
65. Bland JM, Altman DG (1995) Statistics notes: Calculating correlation coefficients with repeated observations: Part 1- correlation within subjects. Br Med J 310: 446.
66. Fisher RA (1920) 012: A Mathematical Examination of the Methods of Determining the Accuracy of an Observation by the Mean Error, and by the Mean Square Error. Mem R Astron Soc 80: 758-770.
67. Newey WK, West KD (1987) A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix. Econometrica 55: 703-708.
68. StataCorp (2019) Stata Statistical Software: Release 16, Stata Corp LLC, College Station, TX, USA.
69. Oken BS, Kaye JA (1992) Electrophysiologic function in the healthy, extremely old. Neurology 42: 519-526.
70. Oken BS, Amen A (2010) Electroencephalography and Evoked Potentials in Geriatrics. In Clinical Neurology of Aging Oxford University Press, USA. p: 113-127.
71. Ros T, Enriquez-Geppert S, Zotev V, Young KD, Wood G, et al. (2020) Consensus on the reporting and experimental design of clinical and cognitive-behavioural neurofeedback studies (CRED-nf checklist). Brain 143: 1674-1685.