The FDA gets aggressive with Covid-19 antibody tests

Chembio’s antibody test is the first to have its authorisation revoked. It might not be the last.
Policy and regulation

The US FDA’s rescinding of emergency use authorisation for Chembio’s Covid-19 antibody test on grounds of inaccuracy caused the company’s stock to plunge more than 60% yesterday. It might also have prompted the makers of other antibody tests with EUAs to wonder whether the agency's action set a precedent. 

The regulator took the action after performing its own investigation into the accuracy of the DPP Covid-19 IgM/IgG system, as Chembio’s test is called. And it is doing the same for the other assays it has authorised. Three so far have been given the all-clear, which leaves 13 authorised antibody tests still potentially vulnerable to being double-checked and found wanting.

The FDA began granting emergency market passes to antibody tests at the start of April, based on accuracy data submitted by the manufacturers; the agency checked this over but did not run separate evaluations of its own. 

In mid-April the agency said it would start performing independent validations in concert with other US government agencies, including the National Cancer Institute. This has been a slow process, however, with reports on only four commercially produced antibody tests with EUAs having so far been made available. Chembio’s is the first to have run into trouble. 

Crunching the numbers

The data Chembio submitted for its test claimed sensitivity of 93.5%, based on correctly identifying 29 of 31 positive samples, and specificity of 94.4%, getting 118 of 125 negative samples right. (These values are for the combined IgM and IgG antibodies; separate values were also given.) Data from an evaluation sponsored by the NCI and performed at the Frederick National Laboratory for Cancer Research found rather different numbers, summarised below. 

Accuracy data on Chembio's antibody test
  Initially submitted by Chembio NCI evaluation
IgM sensitivity 77.4% 57.1%
IgM specificity - 86.2%
IgG sensitivity 87.1% 78.6%
IgG specificity - 91.2%
Combined sensitivity 93.5% 82.1%
Combined specificity 94.4% 81.2%
Combined PPV 46.8% 18.7%
Combined NPV 99.6% 98.9%
PPV and NPV at 5% prevalence. Source: FDA.

In its revocation letter to Chembio the FDA wrote that it was "not reasonable to believe that the test may be effective in detecting antibodies against Sars-CoV-2”, or that its benefits outweighed its risks, including the high rate of false results. 

The FDA also stated in the letter that its current thinking on antibody tests was that they should have a minimum combined sensitivity of 90% and a minimum specificity of 95%. For tests that report specifically IgM and IgG, minimum sensitivity values of 90% and 70% for IgG and IgM respectively are called for. There are also other conditions including the number of samples on which these figures should be calculated. 

So how do the other antibody tests stack up? Of the four assays that have been independently checked the most interesting is Helgen, which the FDA decided was good enough to stay on the market despite a relatively low positive predictive value – the probability that subjects with a positive test result truly have Covid-19 antibodies – of 68%.

Independently evaluated Covid-19 antibody tests with EUAs
Company Test Sensitivity Specificity PPV NPV
Chembio Diagnostic DPP Covid-19 IgM/IgG 82.1% 81.2% 18.7% 98.9%
Euroimmun (Perkinelmer) Anti-Sars-CoV-2 Elisa IgG 90.0% 100% 100% 99.5%
Hangzhou Biotest Biotech RightSign Covid-19 IgG/IgM  100% 100% 100% 100%
Healgen Covid-19 IgG/IgM 100% 97.5% 67.8% 100%
Commercial tests only. PPV & NPV = positive & negative predictive values. PPV and NPV calculated at 5% prevalence. Source: FDA.

It is important to note that the tables above and below are just summaries – they do not include confidence intervals, for instance, or the sizes of the samples on which accuracy values were calculated. When a product tests separately for both IgG and IgM, combined accuracy figures are given. The FDA’s own page on antibody test performance has a fuller picture, including accuracy data for the separate antibodies where appropriate.

As for the 13 commercial tests that have received EUA but have not yet been independently validated, all look good enough to meet the FDA’s minimum standards, though the positive predictive value of Cellex’s test is markedly low at 55%. And no matter how decent these figures look, NCI scientists might come up with different results.

Multisystem inflammatory syndrome in children (MIS-C) and COVID-19

Learn the signs and symptoms of a rare, serious condition called multisystem inflammatory syndrome in children (MIS-C) that's linked to COVID-19, the new coronavirus disease.

 

Though children of all ages can become sick with coronavirus disease 2019 (COVID-19), most kids who are infected typically don't become as sick as adults do. Some children who have an active infection with the virus that causes COVID-19 might not show any signs or symptoms at all.

 

Still, you may have heard about a serious inflammatory syndrome in children, including some teenagers, that appears to be linked to COVID-19. It's called multisystem inflammatory syndrome in children (MIS-C). This syndrome is rare, and most children who have it eventually get better with medical care. But some kids rapidly get worse, to the point where their lives are at risk.

Much remains to be learned about this new and emerging inflammatory syndrome, and the cause is not known yet. But if your child shows any signs or symptoms, get help fast. Here's what you need to know.

What is multisystem inflammatory syndrome in children (MIS-C)?

 

Multisystem inflammatory syndrome in children (MIS-C) is a serious condition in which some parts of the body — such as the heart, blood vessels, kidneys, digestive system, brain, skin or eyes — become inflamed. Inflammation typically includes swelling, often with redness and pain.

 

Many, but not all, children with MIS-C test negative for a current infection with the virus that causes COVID-19. Yet evidence indicates that many of these children were infected with the COVID-19 virus in the past, as shown by positive antibody test results.

An antibody test with a positive result means that the child's immune system developed blood proteins (antibodies) that fought the COVID-19 virus. Sometime this blood test is the only indication that the child was ever infected — meaning the child may have fought the infection without ever having shown signs or symptoms of COVID-19. Still, some children with MIS-C are currently infected with the virus, usually confirmed by detection of the virus on a swab taken from the back of the nose or throat.

MIS-C shares some of the same signs and symptoms as another condition called Kawasaki disease. Kawasaki disease mainly affects children under 5 years of age. It causes inflammation in the walls of blood vessels, particularly those that supply blood to the heart muscle (coronary arteries). Researchers are working to figure out if the two conditions are related or not.

Com sistema instável no Ministério da Saúde, Estado divulga ...

What are the signs and symptoms of MIS-C?

 

Signs and symptoms of multisystem inflammatory syndrome in children include those below, though not all children have the same symptoms:

  • Fever that lasts 24 hours or longer
  • Vomiting
  • Diarrhea
  • Pain in the stomach
  • Skin rash
  • Red eyes
  • Redness or swelling of the lips and tongue
  • Feeling unusually tired
  • Redness or swelling of the hands or feet
 

Emergency warning signs of MIS-C include:

  • Inability to wake up or stay awake
  • Difficulty breathing
  • Chest pain or pressure that doesn't go away
  • New confusion
  • Bluish lips or face
  • Severe stomach pain

What should you do if you think your child has MIS-C?

 

If your child shows any of the emergency warning signs listed above — or is severely sick with other signs and symptoms — get care immediately. Take your child to the nearest emergency department or call 911 or your local emergency number.

If your child isn't severely ill but shows other signs or symptoms of MIS-C, contact your child's doctor right away for advice. Doctors may want to do some tests — such as blood tests, or imaging tests of the chest, heart or abdomen — to check for areas of inflammation and other signs of MIS-C.

 

How is MIS-C treated?

 

Most children with MIS-C need to be treated in a hospital, and some will need treatment in a pediatric intensive care unit. Treatment usually involves different types of therapies that target the immune system and reduce inflammation. Depending on your child's symptoms and condition, he or she may receive anti-inflammatory drugs and other medications to reduce inflammation in the affected vital organs, such as the heart or kidneys, and protect them from permanent damage.

 

MIS-C is not contagious, but there's a chance that your child could have an active infection with the COVID-19 virus or another type of contagious infection. So the hospital will use infection control measures while caring for your child.

How to help prevent your child from getting MIS-C

 

The best way to help prevent your child from getting MIS-C is to take action to avoid exposure to the COVID-19 virus and teach your child how to do the same. Follow the guidelines of the U.S. Centers for Disease Control and Prevention:

 
  • Keep hands clean. Wash hands often with soap and water for at least 20 seconds. If soap and water aren't available, use a hand sanitizer that contains at least 60% alcohol.
  • Avoid people who are sick. In particular, avoid people who are coughing, sneezing or showing other signs and symptoms that indicate they might be sick and contagious.
  • Practice social distancing. This means that you and your child should stay at least 6 feet (2 meters) from other people when outside of your home.
  • Wear cloth face masks in public settings. When it's difficult to practice social distancing, both you and your child — if he or she is age 2 years or older — should wear face masks that cover the nose and mouth.
  • Clean and disinfect high-touch surfaces every day. This includes areas of your home such as doorknobs, light switches, remotes, handles, countertops, tables, chairs, desks, keyboards, faucets, sinks and toilets.
  • Wash clothing and other items as needed. Follow manufacturers' instructions, using the warmest appropriate water setting on your washing machine. Remember to include washable plush toys.

You can be assured that experts continue to collect and analyze data about COVID-19 Test Kit to learn more about this complex condition and its possible causes, and to improve diagnosis, treatment and outcomes for children.

Readmore: Coronavirus disease 2019 (COVID-19)

Using Antibody Tests for COVID-19

Using COVID-19 Antibody Test

CDC has developed interim guidance for how healthcare providers, laboratories, and public health staff should use antibody tests. These tests look for the presence of antibodies, which are proteins made in response to infections. Antibodies are detected in the blood of people who are tested after infection; they show the body’s efforts to fight off a specific infection.

The virus that causes COVID-19 is new, and what we know about it changes rapidly. This guidance will be updated as more information becomes available.

Laboratory scientist holds a serological plate for SARS-CoV-2 antibody testing.
 
alert icon
At-A-Glance Recommendations for Professionals
ballot check solid icon
Interim Guidelines for Clinical and Public Health Settings

Key points:

  • In general, a positive antibody test is presumed to mean a person has been infected with SARS-CoV-2, the virus that causes COVID-19, at some point in the past. It does not mean they are currently infected.
  • Antibodies start developing within 1 to 3 weeks after infection.
  • We currently don’t have enough information yet to say whether someone will definitely be immune and protected from reinfection if they have antibodies to the virus.
  • Healthcare providers who use antibody tests must know how the different tests work and use caution when interpreting test results:
    • If someone tests positive for SARS-CoV-2 antibodies but does not really have those specific antibodies, the result is a false positive. Similarly, if someone tests negative for SARS-CoV-2 antibodies but does really have those specific antibodies, the result is a false negative.
    • FDA has authorized antibody tests for this virus that have been submitted for their review. But these tests are not 100% accurate and some false positive results or false negative results may occur.
    • A higher percentage of positive results may be false positives when these tests are used in people who live or work in an area where very few people have had COVID-19.
  • People who receive positive results on an antibody test but don’t have symptoms of COVID-19 or have not been around someone who may have COVID-19 are not likely to have a current infection. They can continue with normal activities, including work, but still take steps to protect themselves and others.
  • People who receive positive results on an antibody test and who are currently or recently sick or have been around someone with COVID-19 should follow CDC recommendations on caring for themselves and protecting others, and when they can be around other people again.

Readmore: Interim Guidelines for COVID-19 Antibody Testing

 

Do:

  • Until scientists get more data on whether antibodies protect against reinfection with this virus, everyone should continue to take steps to protect themselves and others, including staying at least 6 feet away from other people outside of their home (social distancing), even if they have had a positive antibody test.
  • People who wear personal protective equipment (PPE) at work should continue to wear PPE, even if they test positive for antibodies to the virus.

Don’t:

  • Antibody test results should not be used to determine if someone can return to work.
  • Antibody test results should not be used to group people together in settings such as schools, dormitories, and correctional facilities.

Interim Guidelines for COVID-19 Antibody Testing

Summary

Interim Guidelines for COVID-19 Antibody Testing


Serologic assays for SARS-CoV-2 now have Emergency Use Authorization (EUA) by the U.S. Food and Drug Administration (FDA), which has independently reviewed their performance.Serologic methods have been developed and will have important public health and clinical uses to monitor and respond to the COVID-19 pandemic.

  • Currently, there is no identified advantage of assays whether they test for IgG, IgM and IgG, or total antibody.
  • It is important to minimize false positive test results by choosing an assay with high specificity and by testing populations and individuals with an elevated likelihood of previous exposure to SARS-CoV-2. Alternatively, an orthogonal testing algorithm (i.e., employing two independent tests in sequence when the first test yields a positive result) can be used when the expected positive predictive value of a single test is low.
  • Antibodies most commonly become detectable 1-3 weeks after symptom onset, at which time evidence suggests that infectiousness likely is greatly decreased and that some degree of immunity from future infection has developed. However, additional data are needed before modifying public health recommendations based on serologic test results, including decisions on discontinuing physical distancing and using personal protective equipment.

COVID-19 Testing Project

  Background

Serologic assays for SARS-CoV-2, now broadly available, can play an important role in understanding the virus’s epidemiology in the general population and identifying groups at higher risk for infection. Unlike viral direct detection methods such as nucleic acid amplification or antigen detection tests that can detect acutely infected persons, antibody tests help determine whether the individual being tested was ever infected—even if that person never showed symptoms. Serologic tests detect waning or past SARS-CoV-2 virus infection indirectly, by measuring the host humoral immune response to the virus. Therefore, serology assays do not typically replace direct detection methods as the primary tool for diagnosing an active SARS-CoV-2 infection, but they do have several important applications in monitoring and responding to the COVID-19 pandemic.

Although serologic tests should not be used at this time to determine if an individual is immune, these tests can help determine the proportion of a population previously infected with SARS-CoV-2 and provide information about populations that may be immune and potentially protected. Thus, demographic and geographic patterns of serologic test results can help determine which communities may have experienced a higher infection rate and therefore may have higher rates of herd immunity. In some instances, serologic test results may assist with identifying persons potentially infected with SARS-CoV-2 and determining who may qualify to donate blood that can be used to manufacture convalescent plasmaexternal icon as a possible treatment for those who are seriously ill from COVID-19.

  Development of Antibodies and Immunity

Nearly all immune competent individuals will develop an immune response following SARS-CoV-2 infection. Like infections with other pathogens, SARS-CoV-2 infection elicits development of IgM and IgG antibodies, which are the most useful for assessing antibody response because little is known about IgA response in the blood.

Antibodies in some persons can be detected within the first week of illness onset. SARS-CoV-2 infections are somewhat unusual because IgM and IgG antibodies arise nearly simultaneously in serum within 2 to 3 weeks after illness onset. Thus, detection of IgM without IgG is uncommon. How long IgM and IgG antibodies remain detectable following infection is not known.

In addition, development of neutralizing antibodies can also be assessed. Neutralizing antibodies inhibit viral replication in vitro, and as with many infectious diseases, their presence correlates with immunity to future infection, at least temporarily.

Recurrence of COVID-19 illness appears to be very uncommon, suggesting that the presence of antibodies could confer at least short-term immunity to infection with SARS-CoV-2. Consistent with this observation, experimental primary infection in primates and subsequent development of antibodies resulted in protection from reinfection after the primates were rechallenged. Additionally, antibody development in humans correlates with a marked decrease in viral load in the respiratory tract. Taken together, these observations suggest that the presence of antibodies may decrease a person’s infectiousness and offer some level of protection from reinfection. However, definitive data are lacking, and it remains uncertain whether individuals with antibodies (neutralizing or total) are protected against reinfection with SARS-CoV-2, and if so, what concentration of antibodies is needed to confer protection.

  Current Status of Antibody Testing in the United States

Antigenic targets

The two major antigenic targets of SARS-CoV-2 virus against which antibodies are detected are spike glycoprotein (S) and nucleocapsid phosphoprotein (N). While S protein is essential for virus entry and is present on the viral surface, N protein is the most abundantly expressed immunodominant protein that interacts with RNA. Multiple forms of S protein — full-length (S1+S2) or partial (S1 domain or receptor binding domain [RBD]) — are used as antigens. The protein target determines cross-reactivity and specificity because N is more conserved across coronaviruses than S, and within S, RBD is more conserved than S1 or full-length S.

  Types of Antibody Testing

Different types of assays can be used to determine different aspects of immune response and functionality of antibodies. The tests can be broadly classified to detect either binding or neutralizing antibodies.

  • Binding antibody detection: These tests use purified proteins of SARS-CoV-2, not live virus, and can be performed in lower biosafety level laboratories (e.g., BSL-2). With specific reagents, individual antibody types, like IgG, IgM, and IgA, can be determined. In general, IgM is one of the first types of antibodies produced after infection and is most useful for determining recent infection, while IgG generally develops after IgM and may remain detectable for months or years. IgA is important for mucosal immunity and can be detected in mucous secretions like saliva in addition to blood, though its significance in this disease is still to be determined. Depending on the complexity of assays, these tests can be performed rapidly (less than 30 minutes) in a field setting or in a few hours in a laboratory.

    Tests that detect binding antibodies fall into two broad categories.

    • Point-of-care (POC) tests generally are lateral flow devices that detect IgG or IgG and IgM, or total antibody in serum, plasma, whole blood, and/or saliva. An advantage of some point-of-care tests using whole blood is that they can be performed on blood samples obtained by fingerstick rather than venipuncture.
    • Laboratory tests use ELISA (Enzyme-Linked Immunosorbent Assay) or CIA (chemiluminescent immunoassay) methods for antibody detection, which for some assays may require trained laboratorians and specialized instruments. Based on the reagents, IgG, IgM, and IgA can be detected separately or combined as total antibody.
  • Neutralizing antibody detection: FDA has not yet authorized the use of neutralization tests for SARS-CoV-2. Neutralization tests determine the functional ability of antibodies to prevent infection of virus in vitro. The test involves incubating serum or plasma with live virus followed by infection and incubation of cells. Testing will require either BSL-3 or BSL-2 laboratories, depending on what form of the SARS-CoV-2 virus is used.

    Two types of neutralization tests are conducted.

    • Virus neutralization tests (VNT), such as the plaque-reduction neutralization test (PRNT) and microneutralization, use a SARS-CoV-2 virus from a clinical isolate or recombinant SARS-CoV-2 expressing reporter proteins. This testing requires BSL-3 laboratories and may take up to 5 days to complete.
    • Pseudovirus neutralization tests (pVNT) use recombinant pseudoviruses (like vesicular stomatitis virus, VSV) that incorporate the S protein of SARS-CoV-2. This testing can be performed in BSL-2 laboratories depending on the VSV strain used.

FDA-authorized serologic tests

FDA now requires commercially marketed serologic tests to receive Emergency Use Authorization (EUA)external icon. Tests that are not commercially marketed do not require FDA authorization but developers may voluntarily request authorization. Multiple agencies — including FDA, the National Cancer Institute/National Institutes of Health (NCI/NIH), CDC, and the Biomedical Advanced Research and Development Authority (BARDA) — are collaborating with members of academia and the medical community to evaluate several serology tests using a well-characterized set of clinical samples (serum or plasma) collected before and during the current COVID-19 outbreak. A list of all tests authorized for emergency use under EUA is maintained on an FDA website external icon. All currently authorized tests are qualitative (providing a result that is positive, negative, or indeterminate) rather than quantitative (providing a quantitative assessment of antibody levels).

Both laboratory and rapid serologic assays have received EUA. Serologic testing technologies include single-use, low-throughput lateral flow tests where the presence of antibody is demonstrated by a color change on a paper strip and laboratory-based immunoassays that allow for processing of many samples at the same time.

The EUA letter of authorization includes the settings in which the test is authorized, based on FDA’s determination of appropriate settings for use during the public health emergency.

  Optimizing Testing Outcomes

Test performance

The utility of tests depends on the sensitivity and specificity of the assays; these performance characteristics are determined by using a defined set of negative and positive samples. In addition, the predictive values of a test should be considered because these values affect the overall outcome of testing. Positive predictive value is the probability that individuals with positive test results are truly antibody positive. Negative predictive value is the probability that individuals with negative test results are truly antibody negative. Positive and negative predictive values are determined by the percentage of truly antibody positive individuals in the tested population (prevalence, pre-test probability) and the sensitivity and specificity of the test. For example:

  • In a high-prevalence setting, the positive predictive value increases — meaning it is more likely that persons who test positive are truly antibody positive – than if the test is performed in a population with low-prevalence. When a test is used in a population where prevalence is low, the positive predictive value drops because there are more false-positive results, since the pre-test probability is low.
  • Likewise, negative predictive value is also affected by prevalence. In a high-prevalence setting, the negative predictive value declines whereas in a low-prevalence setting, it increases.

In most of the country, including areas that have been heavily impacted, the prevalence of SARS-CoV-2 antibody is expected to be low, ranging from <5% to 25%, so that testing at this point might result in relatively more false positive results and fewer false-negative results.

In some settings, such as COVID-19 outbreaks in food processing plants and congregate living facilities, the prevalence of infection in the population may be significantly higher. In such settings, serologic testing at appropriate intervals following outbreaks might result in relatively fewer false positive results and more false-negative results.

Testing strategies

In the current pandemic, maximizing specificity and thus positive predictive value in a serologic algorithm is preferred in most instances, since the overall prevalence of antibodies in most populations is likely low. For example, in a population where the prevalence is 5%, a test with 90% sensitivity and 95% specificity will yield a positive predictive value of 49%. In other words, less than half of those testing positive will truly have antibodies. Alternatively, the same test in a population with an antibody prevalence exceeding 52% will yield a positive predictive greater than 95%, meaning that less than one in 20 people testing positive will have a false positive test result.

Three strategies can be used to improve positive predictive value:

  • Choosing a test with a very high specificity, perhaps 99.5% or greater, will yield a high positive predictive value in populations tested with prevalence >5%.
  • Another strategy is to focus testing on persons with a high pre-test probability of having SARS-CoV-2 antibodies, such as persons with a history of COVID-19-like illness.
  • A third approach is to employ an orthogonal testing algorithm in which persons who initially test positive are tested with a second test. Effective orthogonal algorithms are generally based on testing a patient sample with two tests, each with unique design characteristics (e.g., antigens or formats).

Algorithms can be designed to maximize overall specificity while retaining maximum sensitivity. For example, in the example above with a population prevalence of 5%, a positive predictive value of 95% can be achieved if samples initially positive are tested with a second different orthogonal assay that also has 90% sensitivity and 95% specificity. The performance of orthogonal testing algorithms has not been systematically evaluated but can be estimated using an on-line calculator external icon from the FDA. See Table 1 for the potential improvement benefits of the orthogonal testing algorithm.

  Limitations of Serologic Tests

At present, the immunologic correlates of immunity from SARS-CoV-2 infection are not well defined.  Representatives from BARDA, CDC, FDA, NIH, the Office of the Assistant Secretary for Health (OASH), Department of Defense (DoD), and White House Office of Science and Technology Policy (OSTP) are working with members of academia and the medical community to determine whether positive serologic tests are indicative of protective immunity against SARS-CoV-2.  This work includes assessing the level of antibodies required for protection from reinfection, the duration of that protection, and the factors associated with development of a protective antibody response. The kinetics of antibody response, longevity of antibodies, the ability of antibodies to protect from repeat infection, the protective titer of neutralizing antibody, and the correlation of binding antibody titers to neutralization ability are yet to be determined. Although animal challenge studies demonstrate protection in the short run, demonstration of long-term protection in humans will require future study. Hence, pending additional data, the presence of antibodies cannot be equated with an individual’s immunity from SARS-CoV-2 infection.

Some tests may exhibit cross-reactivity with other coronaviruses, such as those that cause the common cold. This could result in false-positive test results. Some persons may not develop detectable antibodies after coronavirus infection. In others, it is possible that antibody levels could wane over time to undetectable levels. IgM and IgG antibodies are not present early in infection. Thus, serologic test results do not indicate with certainty the presence or absence of current or previous infection with SARS-CoV-2.