跳转至

File Transfer Mode Design

Design notes for adding TWAIN File Transfer (TWSX_FILE) support to BN Tech Virtual Scanner alongside the existing Native Transfer (TWSX_NATIVE) path.

1. Requirement

In addition to the existing Native Transfer path, the virtual scanner needs to support TWAIN File Transfer mode, so an application can let the data source write the scanned image to disk and just receive the resulting file path back.

Main requirements:

  • Support ICAP_XFERMECH = TWSX_FILE.
  • Support ICAP_IMAGEFILEFORMAT with PNG / JPG / BMP / TIFF.
  • Support DAT_SETUPFILEXFER with MSG_GET / MSG_GETDEFAULT / MSG_SET / MSG_RESET.
  • Support DAT_IMAGEFILEXFER / MSG_GET performing the actual file write and returning a populated TW_SETUPFILEXFER.
  • Support two file path sources:
  • Application-supplied path via DAT_SETUPFILEXFER / MSG_SET (XnView "Scan to..." etc.).
  • User-selected output directory + filename + format from the settings UI.
  • Hide UI output fields when the application has already taken over file output, to avoid misleading the user and accidentally overwriting the app-supplied path.
  • File DPI metadata (PNG pHYs, JPG JFIF density, BMP biXPelsPerMeter, TIFF XResolution, ...) must match the DPI selected in the settings UI.
  • File extension must match the chosen format (PNG -> .png, JPG -> .jpg, BMP -> .bmp, TIFF -> .tif).
  • Unsupported formats or missing directories must not crash the DS; failure should be reported with a TWAIN condition code.

Non-functional requirements:

  • The flow must follow the TWAIN state machine: enter file write at State 6 (kXferReady), transition to State 7 (kXferring) on success, and end via DAT_PENDINGXFERS / MSG_ENDXFER.
  • Applications may call DAT_SETUPFILEXFER / MSG_SET after MSG_XFERREADY (e.g. TWACK), so the DS must defer the actual file write until DAT_IMAGEFILEXFER / MSG_GET.

2. Domain knowledge

2.1 The three TWAIN transfer mechanisms

TWAIN defines three values for ICAP_XFERMECH:

Value Meaning
TWSX_NATIVE DS returns the whole image as a DIB handle.
TWSX_FILE DS writes the image to disk and returns the file path and format.
TWSX_MEMORY DS returns the image in strips. Not supported in this project.

This project supports TWSX_NATIVE and TWSX_FILE. CAP_XFERMECH defaults to TWSX_NATIVE and can be switched to TWSX_FILE either by the application or in the settings UI.

2.2 TWAIN triples involved in File Transfer

1. DG_CONTROL / DAT_CAPABILITY  / MSG_SET   -> ICAP_XFERMECH = TWSX_FILE
2. DG_CONTROL / DAT_CAPABILITY  / MSG_SET   -> ICAP_IMAGEFILEFORMAT = TWFF_PNG
3. DG_CONTROL / DAT_SETUPFILEXFER / MSG_SET -> Optional path + Format
4. DG_CONTROL / DAT_USERINTERFACE / MSG_ENABLEDS
5. DG_CONTROL / DAT_EVENT / MSG_PROCESSEVENT -> wait MSG_XFERREADY
6. (state 6) Optionally another DAT_SETUPFILEXFER / MSG_SET to update path
7. DG_IMAGE / DAT_IMAGEINFO / MSG_GET
8. DG_IMAGE / DAT_IMAGEFILEXFER / MSG_GET   -> DS writes file, returns path
9. DG_CONTROL / DAT_PENDINGXFERS / MSG_ENDXFER
10. DG_CONTROL / DAT_USERINTERFACE / MSG_DISABLEDS

2.3 TW_SETUPFILEXFER

DAT_SETUPFILEXFER and DAT_IMAGEFILEXFER share TW_SETUPFILEXFER: FileName (max 255 chars), Format (TWFF_*), and the Mac-legacy VRefNum (always 0 here).

2.4 Application-driven vs UI-driven paths

  • App-driven (XnView "Scan to..."): the app pops up its own dialog and supplies the destination path via DAT_SETUPFILEXFER / MSG_SET. The DS must not expose its own output directory fields in this case.
  • UI-driven: the user picks directory, filename, and format in the settings UI; the DS builds the path itself.
  • Mixed (TWACK): the app first calls MSG_ENABLEDS, then sends DAT_SETUPFILEXFER / MSG_SET after the settings UI closes. The DS must defer the file write and must not overwrite the path the app is about to supply.

2.5 DPI metadata

FreeImage_Save is inconsistent about DPI metadata across formats and versions. To guarantee correct DPI in Windows Explorer's Details tab and downstream applications, the project re-patches DPI fields after saving via format-specific patchers (patchPngDpiMetadata, patchJpegDpiMetadata, patchBmpDpiMetadata, patchTiffDpiMetadata).

3. Design goals

  • Cover the full File Transfer state machine: all four DAT_SETUPFILEXFER messages plus DAT_IMAGEFILEXFER / MSG_GET.
  • Support both app-driven and UI-driven path sources via the app_managed_file_output flag flowing into the settings UI.
  • Share the image pipeline (acquireImage + preScanPrep) with Native Transfer; only the final step differs (DIB vs file).
  • Keep file DPI metadata aligned with the UI-selected DPI for PNG / JPG / BMP / TIFF.
  • Leave hooks for future formats (multi-page TIFF, PDF, etc.).

Non-goals:

  • No TWSX_MEMORY.
  • No multi-page output.
  • No encryption/compression negotiation; JPG quality is fixed at 85.
  • No PDF / OCR output.

4. Overall design

File Transfer layers minimally on top of Native Transfer:

TwainDataSource
├── handleDatSetupFileXfer()       // Path negotiation
├── handleDatImageFileXfer()       // Triggers file write + returns path
└── enableDs()                     // Drives settings UI when ShowUI=TRUE
        │
        ├── SettingsServer (HTML UI)
        │     └── app_managed_file_output gates output fields
        │
        └── VirtualScanner
              ├── acquireImage() + preScanPrep()
              ├── saveImageToFile()      // Uses UI dir + filename + format
              ├── saveImageToPath()      // Uses app-supplied full path
              ├── applyDpiMetadata()
              └── patchSavedDpiMetadata()

High-level flow:

  1. App sets ICAP_XFERMECH = TWSX_FILE and ICAP_IMAGEFILEFORMAT.
  2. App optionally calls DAT_SETUPFILEXFER / MSG_SET; the DS records it in app_file_path_ and fixes ICAP_IMAGEFILEFORMAT from the extension.
  3. MSG_ENABLEDS:
  4. If ShowUI=TRUE, show settings UI. When cur_mech == TWSX_FILE, set app_managed_file_output = true and hide output fields.
  5. Call acquireImage() to prepare the pixels but do not write the file.
  6. Emit MSG_XFERREADY.
  7. App may call DAT_SETUPFILEXFER / MSG_SET again after MSG_XFERREADY.
  8. On DAT_IMAGEFILEXFER / MSG_GET:
  9. Non-empty app_file_path_ -> saveImageToPath().
  10. Otherwise -> saveImageToFile() using UI settings.
  11. Populate data->FileName / Format / VRefNum, move to kXferring, return TWRC_XFERDONE.
  12. App ends the transfer with DAT_PENDINGXFERS / MSG_ENDXFER. closeDs() clears app_file_path_.

5. Key decisions and rationale

5.1 Defer file write until DAT_IMAGEFILEXFER / MSG_GET

enableDs() only calls acquireImage(). The actual save happens later, because the spec allows DAT_SETUPFILEXFER / MSG_SET in State 6, after MSG_XFERREADY, and tools like TWACK use this pattern.

5.2 Both saveImageToFile() and saveImageToPath()

App-driven scenarios must write exactly to the path the app supplied. UI-driven scenarios need the DS to invent a timestamped filename under a user-selected directory. Two entry points keep both code paths simple; both share applyDpiMetadata + patchSavedDpiMetadata.

5.3 app_managed_file_output flag in the settings UI

When enableDs() sees ICAP_XFERMECH == TWSX_FILE, it tells the settings UI to hide all output controls. This prevents the user from changing fields that won't be honored (the app owns the path) and avoids accidentally overwriting app_file_path_.

5.4 Extension-based format fix-up in DAT_SETUPFILEXFER / MSG_SET

Some applications send Format = 0 or always send TWFF_PNG. The DS derives the format from the file extension and updates ICAP_IMAGEFILEFORMAT, so the written file matches its extension and later MSG_GET queries return a consistent format.

5.5 Post-save DPI metadata patching

FreeImage's DPI handling is uneven across formats and versions. Re-patching after FreeImage_Save guarantees correct horizontal/vertical DPI in Explorer and other consumers.

5.6 MSG_RESET clears app_file_path_

MSG_RESET is treated as "no app-supplied path, fall back to UI-driven". closeDs() clears it too, so the path does not leak across sessions.

6. Component changes

6.1 capability.cpp

  • Add TWSX_FILE to ICAP_XFERMECH choices.
  • Add ICAP_IMAGEFILEFORMAT (default TWFF_PNG) with TWFF_PNG / TWFF_JFIF / TWFF_BMP / TWFF_TIFF.
  • Expose both in CAP_SUPPORTEDCAPS.

6.2 twain_data_source.h / .cpp

  • New member std::string app_file_path_.
  • New dispatch:
  • DAT_SETUPFILEXFER -> handleDatSetupFileXfer()
  • DAT_IMAGEFILEXFER -> handleDatImageFileXfer()
  • handleDatSetupFileXfer:
  • MSG_SET: store the path; auto-correct ICAP_IMAGEFILEFORMAT from the extension.
  • MSG_GET / MSG_GETDEFAULT: return current app_file_path_ and ICAP_IMAGEFILEFORMAT.
  • MSG_RESET: clear app_file_path_ then fall through to MSG_GET.
  • handleDatImageFileXfer:
  • Requires kXferReady.
  • Branches between saveImageToPath() and saveImageToFile().
  • Fills FileName / Format / VRefNum, transitions to kXferring, returns TWRC_XFERDONE.
  • enableDs():
  • Sets app_managed_file_output = true when ShowUI=TRUE and ICAP_XFERMECH == TWSX_FILE.
  • Writes UI choices back into capabilities and into VirtualScanner.
  • Always calls acquireImage() before MSG_XFERREADY, regardless of transfer mode.
  • closeDs() resets app_file_path_.

6.3 virtual_scanner.h / .cpp

  • Adds output_dir_ / output_format_ / output_filename_ / last_saved_file_ and setters.
  • Adds saveImageToFile(), saveImageToPath(path), and getLastSavedFilePath().
  • Ensures the destination directory exists via SHCreateDirectoryExA.
  • Maps ICAP_IMAGEFILEFORMAT index -> FREE_IMAGE_FORMAT and .ext.
  • Calls applyDpiMetadata() before save and patchSavedDpiMetadata() after.

6.4 settings_server.cpp (HTML UI)

  • SettingsUiResult adds transfer_mode / file_format / output_dir / output_filename / app_managed_file_output.
  • The HTML renders the output group only when app_managed_file_output == false.
  • JS hides/shows format / output rows based on the transfer mode selection.
  • A /browse endpoint hosts SHBrowseForFolderW for picking the output directory.

6.5 File DPI patchers

patchPngDpiMetadata, patchJpegDpiMetadata, patchBmpDpiMetadata, and patchTiffDpiMetadata rewrite the container-level DPI fields after FreeImage_Save. They are reused by File Transfer and by any Native Transfer flow where the application saves the file.

7. Typical flows

7.1 XnView "Scan to..." (app-driven)

1. App: ICAP_XFERMECH = TWSX_FILE
2. App: DAT_SETUPFILEXFER / MSG_SET, FileName="D:\out\page.tif", Format=TWFF_TIFF
3. App: MSG_ENABLEDS, ShowUI=TRUE
   DS:  app_managed_file_output=true -> UI hides output group
        User picks 600 DPI / RGB -> Scan
   DS:  acquireImage(), MSG_XFERREADY
4. App: DAT_IMAGEINFO / MSG_GET -> DS returns 600 DPI image info
5. App: DAT_IMAGEFILEXFER / MSG_GET
   DS:  saveImageToPath("D:\out\page.tif") + patchTiffDpiMetadata
        returns FileName / Format=TWFF_TIFF, TWRC_XFERDONE
6. App: DAT_PENDINGXFERS / MSG_ENDXFER -> MSG_DISABLEDS

7.2 UI-driven

1. App: MSG_ENABLEDS, ShowUI=TRUE
   DS:  XFERMECH currently TWSX_NATIVE -> UI shows full output group
        User picks File / PNG / D:\scans\ / default timestamp -> Scan
   DS:  setCurrentValue(ICAP_XFERMECH, TWSX_FILE)
        setCurrentValue(ICAP_IMAGEFILEFORMAT, TWFF_PNG)
        scanner_.setOutputDir / setOutputFormat / setOutputFilename
        acquireImage(), MSG_XFERREADY
2. App: DAT_IMAGEFILEXFER / MSG_GET
   DS:  app_file_path_ empty -> saveImageToFile()
        writes D:\scans\scan_20260526_153012.png
        patchPngDpiMetadata -> returns path + Format=TWFF_PNG
3. App: MSG_ENDXFER -> MSG_DISABLEDS

8. Limitations

  • TWSX_MEMORY strip mode is not implemented; legacy apps that only support Memory mode will not work.
  • No multi-page output (multi-page TIFF / PDF). pending_xfers_.Count is fixed to 1.
  • JPG quality is hard-coded at 85; there is no quality-related capability negotiation.
  • The ICAP_IMAGEFILEFORMAT <-> output_format_ mapping is duplicated between twain_data_source.cpp and virtual_scanner.cpp; adding a format requires updating both.
  • saveImageToPath infers format from extension only; missing or misspelled extensions silently fall back to PNG.
  • No TW_SETUPFILEXFER2, so paths are capped at 255 characters and silently truncated beyond that.
  • File writes are synchronous on the TWAIN thread; large images on slow disks block the message loop briefly.
  • On write failure only TWCC_BUMMER is returned; disk-full, permission, and missing-directory cases are not distinguished.
  • app_managed_file_output is decided at enableDs() time; an app that switches ICAP_XFERMECH only after MSG_ENABLEDS would see a UI that does not match the final mode.

9. Next steps

  • Evaluate and implement TWSX_MEMORY strip transfer to cover Memory-only applications.
  • Add multi-page TIFF / PDF output and allow pending_xfers_.Count > 1, in preparation for a future simulated ADF.
  • Expose JPG quality and PNG compression level either through settings UI or via capability negotiation.
  • Move file writes off the TWAIN thread and surface progress in the settings UI.
  • Consolidate the format mapping (ICAP_IMAGEFILEFORMAT <-> FREE_IMAGE_FORMAT <-> extension <-> UI index) into a single table.
  • Improve saveImageToPath fallback: use current ICAP_IMAGEFILEFORMAT when the extension is unrecognized, instead of hard-coding PNG.
  • Return finer-grained condition codes for disk full, permission denied, or directory creation failure.
  • Add TW_SETUPFILEXFER2 support for long paths.
  • Integration-test with TWACK, XnView, NAPS2, and similar applications; document the DAT_SETUPFILEXFER / DAT_IMAGEFILEXFER ordering each application uses.
  • Provide a more visible cue in the settings UI when File Transfer is app-managed (icon + read-only path preview).