ABI versus SCF - Why SCF is better than ABI?

[admin]

SCF is a new format with more advantages (open format, better planning for future modifications/versions, better precision, smaller size, etc) than ABI.
Many sequencing machine that are using old software may generate invalid/exotic ABI files. This might cause loss or corruption of data in these files.

Temporary solution:
DNA Baser Assembler can automatically restore the corrupted data in most cases. If you want to see if your ABI files are corrupted/non-standard, open them in DNA Baser Assembler. Then close the file and look for a log file called “Corrupted Abi detected.log.txt”. In this log file, you will find information about what was wrong in your file(s) and how DNA Baser worked to recover your data.
Note: this file may not be generated if the \"Generate report when corrupted ABI files were detected\" option is disabled.

Permanent solution:
Please upgrade your machine software to generate the results in SCF format. For Applied Biosystems users there is a free software upgrade from v5 to v5.2 available on the site.
This upgrade will also allow your machine to produce SCF files with QV included.
SCF files have a smaller chance to become corrupted.

[admin]

This post contains technical details and evidence of the corruption in the ABI files. You don't have to read it. Just use SCF files instead of ABI.

____________________________________________

Some ABI files may have a difference between the numbers of bases and the number of QV records. This means that you may have one or more bases without information about its quality value OR you may have quality values for inexistent bases (dangling QVs).

Starting with version 2.7, DNA Baser can handle these invalid files. A QV equal with zero will be assigned to the dangling bases. This means that even if the peak for that base is good, it will be treated as a bad quality base.


Example of invalid files


Example of correct file. All records have the same size.


In addition, we have found several times ABI files containing invalid information about the QV assigned to the bases. All the QV were zero or one while the range should be between 0 and 255.
Starting with version 2.2, DNA Baser will treat these files as not containing QV since reading/trusting the QV information will render the whole bases in that file to be treated as bad/low quality which will automatically remove your file from contig.



While other programs that opens ABI files tells you nothing about this error, we warn you about this problem and about more possible (undiscovered) errors in this format.
We should think about switching to SCF format.

Please understand this is not a bug in DNA Baser but in ABI file format or in the machine that generate this format (for example the ABI Prism sequencer).