Skip to content

Native Transfer Mode Design

Design notes for the TWAIN Native Transfer (TWSX_NATIVE) path in BN Tech Virtual Scanner.

1. Requirement

The virtual scanner must support TWAIN's default Native Transfer mode (ICAP_XFERMECH = TWSX_NATIVE): the DS assembles the entire image into a single DIB block and hands it to the application as a TWAIN handle in one shot.

Main requirements:

  • ICAP_XFERMECH defaults to TWSX_NATIVE; an app that performs no capability negotiation must still scan via Native Transfer.
  • Support DG_IMAGE / DAT_IMAGENATIVEXFER / MSG_GET, returning a TW_HANDLE lockable with GlobalLock / DSM_MemLock.
  • DIB content = BITMAPINFOHEADER + palette (if applicable) + pixel data; bottom-up rows; each row padded to 4 bytes.
  • Support 1-bit (BW), 8-bit (grayscale), and 24-bit (RGB):
  • 1-bit with a 2-entry palette.
  • 8-bit with a 256-entry grayscale palette.
  • 24-bit without palette; BGR pixel order.
  • TW_IMAGEINFO must match the DIB exactly.
  • Memory must come from the DSM's allocator; fall back to GlobalAlloc / GlobalLock only when the DSM did not supply one.
  • DIB's biXPelsPerMeter / biYPelsPerMeter and TW_IMAGEINFO.XResolution / YResolution must stay in sync.
  • ICAP_UNITS = TWUN_INCHES.
  • Follow the TWAIN state machine: kEnabled (5) -> kXferReady (6) -> kXferring (7) -> kEnabled (5).

Non-functional:

  • Flatbed only; pending_xfers_.Count fixed at 1.
  • Large images must work; no intermediate buffer can fail its single allocation.
  • Share acquireImage + preScanPrep with File Transfer.

2. Domain knowledge

2.1 Protocol sequence

1. MSG_ENABLEDS -> acquireImage(), MSG_XFERREADY (5 -> 6)
2. DAT_EVENT / MSG_PROCESSEVENT
3. DAT_IMAGEINFO / MSG_GET
4. DAT_IMAGENATIVEXFER / MSG_GET -> TW_HANDLE, TWRC_XFERDONE (6 -> 7)
5. DAT_PENDINGXFERS / MSG_ENDXFER (7 -> 5)
6. MSG_DISABLEDS (5 -> 4)

Return codes: TWRC_XFERDONE, TWRC_CANCEL, TWCC_SEQERROR, TWCC_LOWMEMORY.

2.2 DIB layout

BITMAPINFOHEADER -> palette -> pixel data, bottom-up, 4-byte row alignment. 24-bit pixels are BGR. RGBQUAD is {B, G, R, reserved}.

2.3 DSM memory interface

TWAIN 2.4+ supplies DSM_MemAllocate / Free / Lock / Unlock via DAT_ENTRYPOINT / MSG_SET. Use them for any handle returned to the app; fall back to GlobalAlloc / GlobalLock for legacy DSMs.

2.4 TW_IMAGEINFO

XResolution / YResolution (FIX32 DPI), ImageWidth / ImageLength, SamplesPerPixel (1 or 3), BitsPerSample[8] (8 per sample), BitsPerPixel (1/8/24), Planar = FALSE, PixelType, Compression = TWCP_NONE.

2.5 DPI vs biXPelsPerMeter

pixels_per_meter = dpi * 39.37. getImageInfo and allocAndFillDibHeader both derive from the same ScannerSettings.

3. Design goals

  • 100% compatibility with default Native Transfer for TWAIN 2.x apps.
  • Share VirtualScanner pipeline with File Transfer.
  • Strict DSM memory ownership.
  • DIB layout that passes Windows and third-party image-library inspection.
  • 1-bit / 8-bit / 24-bit support.
  • Strip-based pixel reads as the internal abstraction.

Non-goals: TWSX_MEMORY, high-bit-depth pixels, Planar = TRUE, native-side compression, file writes.

4. Overall design

TwainDataSource
├── handleDatImageNativeXfer()  // protocol entry
│     └── transfer()            // strip-based copy into image_data_
│     └── getDibImage()         // wrap into DIB handle
│           ├── allocAndFillDibHeader()
│           └── copyDibPixelData()
├── handleDatImageInfo() / getImageInfo()
└── enableDs()                   // acquireImage + MSG_XFERREADY

VirtualScanner
├── acquireImage()
├── preScanPrep()
├── applyDpiMetadata()
└── getScanStrip()

Key timing:

  1. MSG_ENABLEDS -> acquireImage() -> MSG_XFERREADY, state 5 -> 6.
  2. DAT_IMAGEINFO / MSG_GET -> getImageInfo().
  3. DAT_IMAGENATIVEXFER / MSG_GET -> transfer() + getDibImage(), state 6 -> 7, TWRC_XFERDONE.
  4. DAT_PENDINGXFERS / MSG_ENDXFER -> state 7 -> 5.

5. Key decisions and rationale

5.1 acquireImage() runs immediately in enableDs()

Apps call DAT_IMAGEINFO right after MSG_XFERREADY and need accurate dimensions/bit depth. Decoding up front keeps the state boundary identical to File Transfer.

5.2 Strip-based pixel reads via getScanStrip()

transfer() loops on ~64000-byte strips (rounded to whole rows). Mirrors real scanners, makes future TWSX_MEMORY straightforward, and stays within historically safe DSM bounds.

5.3 Prefer DSM memory functions over GlobalAlloc

Cross-process memory must come from the DSM allocator. Centralized in a small shim that falls back to GlobalAlloc for legacy DSMs.

5.4 Separate lock/unlock for header, palette, and pixels

MemLock does not guarantee stable pointers across calls. Multiple lock/unlock pairs are safe across DSMs and keep error handling localized.

5.5 Skip R/B swap for 24-bit pixels

FreeImage uses BGR on Windows, matching DIB. Skipping a swap saves a full-image pass (about 100 MB at A4 @ 600 DPI).

5.6 Bottom-up rows + 4-byte alignment

Matches Windows DIB convention; misalignment causes downstream readers to misinterpret subsequent rows. BYTES_PERLINE centralizes the alignment math.

5.7 Compute TW_IMAGEINFO on demand

Avoids cache vs state drift; cost is trivial.

5.8 Tolerate apps that skip MSG_PROCESSEVENT

Auto-promote state 5 -> 6 when pending > 0, return TWRC_CANCEL when pending == 0. More forgiving than failing the call.

6. Component changes

6.1 capability.cpp

  • ICAP_XFERMECH default TWSX_NATIVE with choices {TWSX_NATIVE, TWSX_FILE}.
  • ICAP_PIXELTYPE: TWPT_BW / TWPT_GRAY / TWPT_RGB.
  • ICAP_XRESOLUTION / ICAP_YRESOLUTION: 150 / 200 / 300 / 600.
  • ICAP_UNITS = TWUN_INCHES.
  • ICAP_PIXELFLAVOR = TWPF_CHOCOLATE.
  • CAP_UICONTROLLABLE = TRUE.

6.2 twain_data_source.h / .cpp

  • Members: pending_xfers_, image_info_, image_data_ (DSM-allocated intermediate pixel buffer), canceled_, xfer_pending_.
  • Entries: handleDatImageInfo, handleDatImageNativeXfer, handleDatPendingXfers.
  • Helpers: transfer, getDibImage, allocAndFillDibHeader, copyDibPixelData.
  • State transitions: enableDs 5->6, handleDatImageNativeXfer 6->7, endXfer 7->5.
  • DSM memory shim: dsmAlloc / dsmFree / dsmLockMemory / dsmUnlockMemory.

6.3 virtual_scanner.h / .cpp

  • acquireImage(): FreeImage loader.
  • preScanPrep() chain to ensure pixel type matches ICAP_PIXELTYPE.
  • applyDpiMetadata() writes dots-per-meter so TW_IMAGEINFO and DIB header agree.
  • getScanStrip() bottom-up + 4-byte padded.

6.4 settings UI (settings_server.cpp)

  • Native is the default transfer_mode = 0.
  • UI choices flow into ICAP_PIXELTYPE / ICAP_XRESOLUTION / ICAP_YRESOLUTION.
  • File-mode fields are hidden in Native mode.

6.5 DSM interface shim

  • callDsmEntry() dynamic-loads TWAIN_32.dll and caches DSM_Entry.
  • setEntryPoints() stores DSM memory function pointers.
  • Falls back to GlobalAlloc / GlobalLock for legacy DSMs.

7. Typical flows

7.1 Native Transfer with ShowUI=TRUE

1. MSG_OPENDS -> state 4 -> 5
2. ICAP_PIXELTYPE / MSG_SET = TWPT_RGB
3. MSG_ENABLEDS, ShowUI=TRUE
   DS: UI -> RGB / 300 DPI / A4 -> Scan
       acquireImage + preScanPrep
       MSG_XFERREADY, state 5 -> 6
4. DAT_EVENT / MSG_PROCESSEVENT
5. DAT_IMAGEINFO / MSG_GET -> 2480x3508, BPP=24, 300 DPI
6. DAT_IMAGENATIVEXFER / MSG_GET -> TW_HANDLE, TWRC_XFERDONE, 6 -> 7
7. MSG_ENDXFER -> MSG_DISABLEDS

7.2 Native Transfer with ShowUI=FALSE

1. ICAP_PIXELTYPE / MSG_SET = TWPT_GRAY
   ICAP_XRESOLUTION / MSG_SET = 600
2. MSG_ENABLEDS, ShowUI=FALSE
   DS: acquireImage + preScanPrep, MSG_XFERREADY
3. DAT_IMAGEINFO / MSG_GET -> PixelType=TWPT_GRAY, BPP=8, 600 DPI
4. DAT_IMAGENATIVEXFER / MSG_GET -> DIB with 256-entry grayscale palette
5. MSG_ENDXFER -> MSG_DISABLEDS

8. Limitations

  • pending_xfers_.Count fixed at 1.
  • Whole image held in memory (about 100 MB for A4 @ 600 DPI 24-bit).
  • No Planar = TRUE, no compression.
  • No high-bit-depth pixel types.
  • GlobalAlloc fallback has different cross-process semantics from DSM_MemAllocate.
  • Strip size fixed at ~64 KB and not exposed to the app.
  • Auto-promote 5 -> 6 is a compatibility shim, non-standard.
  • BitsPerSample[8] is filled only up to SamplesPerPixel.
  • No progress callback during strip copy.

9. Next steps

  • Add TWSX_MEMORY for very large scans.
  • Support high-bit-depth pixel types (16-bit grayscale, 48-bit RGB).
  • Expose strip size (e.g. via DAT_SETUPMEMXFER or settings UI).
  • Populate XNativeResolution / YNativeResolution.
  • Add a transfer progress callback for the settings UI.
  • Add automated tests with a stub DSM and bit-exact DIB comparisons.
  • Defensive image_data_ cleanup outside closeDs.
  • Runtime-detect FreeImage pixel order instead of relying on a comment.
  • Log when falling back to GlobalAlloc so legacy-DSM memory issues are easier to diagnose.