src/app/core/basic-datatypes/entity/entity.datatype.ts

Description

Datatype for the EntitySchemaService to handle a single reference to another entity. Stored as a simple ID string.

Example:

@DatabaseField({dataType: 'entity', additional: 'Child'}) relatedEntity: string;

Extends

StringDatatype

Relationships

Used by

Depends on

Index

Properties
Methods

Methods

Async anonymize
anonymize(value: unknown, schemaField: EntitySchemaField, parent: unknown)
Inherited from DefaultDatatype

An entity reference points to an independent record, which must not be affected by anonymizing the record that holds the reference. The reference is removed instead, as for any other datatype that does not support partial anonymization.

(this overrides StringDatatype, which would otherwise retain the first character of the id)

Parameters :
Name Type Optional
value unknown No
schemaField EntitySchemaField No
parent unknown No
Returns : Promise<string | undefined>
getExportColumns
getExportColumns(schemaField: EntitySchemaField)
Inherited from DefaultDatatype
Parameters :
Name Type Optional
schemaField EntitySchemaField No
Returns : ExportColumnMapping[]
Async importMapFunction
importMapFunction(val: any, schemaField: EntitySchemaField, additional?: any, importProcessingContext?: ImportProcessingContext)
Inherited from DefaultDatatype

Resolve a single import value to the id of the referenced entity, looking that entity up by the property given in additional.refField.

This is relevant when an entity reference is matched through another entity reference (e.g. import a Note's related Child, identified by the name of the user linked in the Child's "responsible user" field). The nested value has to be resolved to an id before it can be compared with the stored reference.

Parameters :
Name Type Optional
val any No
schemaField EntitySchemaField No
additional any Yes
importProcessingContext ImportProcessingContext Yes
Returns : Promise<string>

the id of the single matching entity or undefined if there is no unique match

Async importMatchField
importMatchField(schemaField: EntitySchemaField, columns: ColumnImportInput[], importProcessingContext: ImportProcessingContext)
Inherited from DefaultDatatype

Matches an import row to actual entities in the database (per-field entry point, see DefaultDatatype.importMatchField).

Splits every mapped column's cell into individual values, resolves any per-column value mapping, then searches candidate entities for each combination of one value per column:

Case target field isArray===false: simple: IMPORT: { x: "x1", y: "y1" } --> matches { x: "x1", y: "y1" } (if there is a single unique match) complex, multi-value import: IMPORT: { x: "x1,x2", y: "y1,y2" } --> matches { x1, y1 } OR { x2, y2 } OR { x1, y2 } OR { x2, y1 } (if there is a single combination that matches)

Case target field isArray===true: simple: IMPORT: { x: "x1", y: "y1" } --> matches every record with { x: "x1", y: "y1" } complex, multi-value import: IMPORT: { x: "x1,x2", y: "y1,y2" } --> matches every record for any combination { x1, y1 }, { x2, y2 }, { x1, y2 }, { x2, y1 }

all matches (isArray=true), or undefined / [] if nothing matched

Parameters :
Name Type Optional Description
schemaField EntitySchemaField No

the target field the value(s) are imported into

columns ColumnImportInput[] No

every column mapped to this field, with its raw cell value

importProcessingContext ImportProcessingContext No

context shared across columns and rows

Returns : Promise<string | [] | undefined>

the id of the single match (isArray=false) or the deduped ids of all matches (isArray=true), or undefined / [] if nothing matched

transformToDatabaseFormat
transformToDatabaseFormat(value: unknown)
Inherited from DefaultDatatype
Parameters :
Name Type Optional
value unknown No
Returns : any
transformToObjectFormat
transformToObjectFormat(value: unknown)
Inherited from DefaultDatatype
Parameters :
Name Type Optional
value unknown No
Returns : any
Static detectAllFieldsInEntity
detectAllFieldsInEntity(entityOrType: Entity | EntityConstructor, dataTypes: string | string[])
Inherited from DefaultDatatype

Detect all fields of the given datatype(s) in an entity's schema.

Parameters :
Name Type Optional Description
entityOrType Entity | EntityConstructor No

An entity instance or entity constructor to inspect.

dataTypes string | string[] No

One or more datatype identifiers to match against.

Returns : literal type[]

Array of matching fields with their id and schema definition.

Static detectFieldInEntity
detectFieldInEntity(entityOrType: Entity | EntityConstructor, dataTypes: string | string[])
Inherited from DefaultDatatype

Detect the first field of the given datatype(s) in an entity's schema.

Scans the schema for a field whose dataType matches one of the provided strings and returns its property name.

Subclasses typically override this without the extra dataTypes parameter, forwarding their own relevant datatype identifiers.

Parameters :
Name Type Optional Description
entityOrType Entity | EntityConstructor No

An entity instance or entity constructor to inspect.

dataTypes string | string[] No

One or more datatype identifiers to match against.

Returns : string | undefined

The field name of the first matching field, or undefined if none is found.

normalizeSchemaField
normalizeSchemaField(schemaField: EntitySchemaField)
Inherited from DefaultDatatype

Return the (potentially adjusted) schema field for this datatype.

Called when schema fields are set up (e.g. from config), allowing the datatype to normalize or fill in required settings.

Override this in a subclass to enforce constraints (e.g. always setting isArray: true).

Parameters :
Name Type Optional Description
schemaField EntitySchemaField No

The current schema field definition

Returns : EntitySchemaField

The schema field to use (default: unchanged)

sortValue
sortValue(_fieldValue: EntityType)
Inherited from DefaultDatatype

Return a comparable primitive for sorting this field's value in a list column. Return undefined to fall through to the default sort logic. Override this in datatypes that store arrays or complex objects where the raw value cannot be sorted meaningfully.

Parameters :
Name Type Optional
_fieldValue EntityType No
Returns : number | string | undefined

Properties

Static dataType
Type : string
Default value : "entity"
Inherited from DefaultDatatype
editComponent
Type : string
Default value : "EditEntity"
Inherited from DefaultDatatype
importAllowsMultiMapping
Type : unknown
Default value : true
Inherited from DefaultDatatype
importConfigComponent
Type : string
Default value : "EntityImportConfig"
Inherited from DefaultDatatype
Static label
Type : string
Default value : $localize`:datatype-label:link to another record`
Inherited from DefaultDatatype
viewComponent
Type : string
Default value : "DisplayEntity"
Inherited from DefaultDatatype
Optional importConfigDialog
Type : string
Inherited from DefaultDatatype

A dialog component in which the import transformation is configured (see MappingDialogData for the data it receives).

Datatypes that require such a dialog get it opened automatically as soon as the user maps a column to one of their fields, so that the suggested settings are reviewed and confirmed rather than silently skipped.

import { inject, Injectable } from "@angular/core";
import { StringDatatype } from "../string/string.datatype";
import { EntitySchemaField } from "../../entity/schema/entity-schema-field";
import { EntityMapperService } from "../../entity/entity-mapper/entity-mapper.service";
import { Logging } from "app/core/logging/logging.service";
import { ImportProcessingContext } from "../../import/import-processing-context";
import { splitArrayValue } from "../../import/split-array-value";
import { ColumnMapping } from "../../import/column-mapping";
import { EntitySchemaService } from "../../entity/schema/entity-schema.service";
import { Entity } from "../../entity/model/entity";
import {
  ColumnImportInput,
  ExportColumnMapping,
} from "../../entity/default-datatype/default.datatype";
import { EntityRegistry } from "../../entity/database-entity.decorator";
import { asArray } from "../../../utils/asArray";

/**
 * Datatype for the EntitySchemaService to handle a single reference to another entity.
 * Stored as a simple ID string.
 *
 * Example:
 *
 * `@DatabaseField({dataType: 'entity', additional: 'Child'}) relatedEntity: string;`
 */
@Injectable()
export class EntityDatatype extends StringDatatype {
  private entityMapper = inject(EntityMapperService);
  private schemaService = inject(EntitySchemaService);
  private readonly entityRegistry = inject(EntityRegistry);

  static override dataType = "entity";
  static override label: string = $localize`:datatype-label:link to another record`;
  override editComponent = "EditEntity";
  override viewComponent = "DisplayEntity";
  override importConfigComponent = "EntityImportConfig";
  override importAllowsMultiMapping = true;

  override getExportColumns(
    schemaField: EntitySchemaField,
  ): ExportColumnMapping[] {
    if (!schemaField.label) {
      return [];
    }

    return [
      {
        keySuffix: "",
        label: schemaField.label,
        resolveValue: (value) => value,
      },
      {
        keySuffix: "_readable",
        label: schemaField.label + " (readable)",
        resolveValue: async (value: string | string[]) =>
          this.loadRelatedEntitiesToString(value, schemaField),
      },
    ];
  }

  /**
   * Matches an import row to actual entities in the database
   * (per-field entry point, see DefaultDatatype.importMatchField).
   *
   * Splits every mapped column's cell into individual values, resolves any
   * per-column value mapping, then searches candidate entities for each
   * combination of one value per column:
   *
   * Case target field isArray===false:
   * simple:
   *  IMPORT: { x: "x1", y: "y1" }
   *  --> matches { x: "x1", y: "y1" } (if there is a single unique match)
   * complex, multi-value import:
   *  IMPORT: { x: "x1,x2", y: "y1,y2" }
   *  --> matches { x1, y1 } OR { x2, y2 } OR { x1, y2 } OR { x2, y1 } (if there is a single combination that matches)
   *
   * Case target field isArray===true:
   * simple:
   *  IMPORT: { x: "x1", y: "y1" }
   *  --> matches every record with { x: "x1", y: "y1" }
   * complex, multi-value import:
   *  IMPORT: { x: "x1,x2", y: "y1,y2" }
   *  --> matches every record for any combination { x1, y1 }, { x2, y2 }, { x1, y2 }, { x2, y1 }
   *
   * @param schemaField the target field the value(s) are imported into
   * @param columns every column mapped to this field, with its raw cell value
   * @param importProcessingContext context shared across columns and rows
   * @returns the id of the single match (isArray=false) or the deduped ids of
   *   all matches (isArray=true), or undefined / [] if nothing matched
   */
  override async importMatchField(
    schemaField: EntitySchemaField,
    columns: ColumnImportInput[],
    importProcessingContext: ImportProcessingContext,
  ): Promise<string | string[] | undefined> {
    const context = new EntityFieldImportContext(
      importProcessingContext,
      schemaField,
    );
    await this.loadImportMapEntities(schemaField.additional, context);
    const candidates = context.entities;

    const criteria = await this.buildMatchCriteria(
      columns,
      context,
      importProcessingContext,
    );
    if (criteria.length === 0) {
      return schemaField.isArray ? [] : undefined;
    }

    // A candidate matches when, for every mapped column, its referenced field
    // value is one of that column's (possibly multiple) values. A mapped column
    // with no value yields an empty list that matches nothing, so an incomplete
    // row cannot match.
    const matchedIds = candidates
      .filter((entity) =>
        criteria.every((criterion) =>
          criterion.values.includes(normalizeValue(entity[criterion.refField])),
        ),
      )
      .map((entity) => entity._id);
    const uniqueIds = [...new Set(matchedIds)];

    if (schemaField.isArray) {
      return uniqueIds;
    }
    if (uniqueIds.length === 1) {
      return uniqueIds[0];
    }
    if (uniqueIds.length > 1) {
      Logging.debug(
        "No unique match found in EntityDatatype importMatchField",
        uniqueIds.length,
      );
    }
    return undefined;
  }

  /**
   * Resolve a single import value to the id of the referenced entity,
   * looking that entity up by the property given in `additional.refField`.
   *
   * This is relevant when an entity reference is matched through another entity
   * reference (e.g. import a Note's related Child, identified by the name of the
   * user linked in the Child's "responsible user" field). The nested value has to
   * be resolved to an id before it can be compared with the stored reference.
   *
   * @returns the id of the single matching entity or undefined if there is no unique match
   */
  override async importMapFunction(
    val: any,
    schemaField: EntitySchemaField,
    additional?: any,
    importProcessingContext?: ImportProcessingContext,
  ): Promise<string> {
    const config = normalizeEntityAdditional(additional);
    if (!config?.refField || !importProcessingContext) {
      return super.importMapFunction(
        val,
        schemaField,
        additional,
        importProcessingContext,
      );
    }

    const match = await this.importMatchField(
      // the caller compares against a single stored value, so never return an array here
      { ...schemaField, isArray: false },
      [
        {
          mapping: {
            column: "",
            propertyName: schemaField.id,
            // the value has already been split by the calling column
            additional: { ...config, enableSplitting: false },
          },
          rawCell: val,
        },
      ],
      importProcessingContext,
    );
    return match as string;
  }

  /**
   * Build one match criterion per mapped column: the referenced field to
   * compare and the acceptable (normalized) values parsed from the column cell.
   */
  private async buildMatchCriteria(
    columns: ColumnImportInput[],
    context: EntityFieldImportContext,
    importProcessingContext: ImportProcessingContext,
  ): Promise<EntityMatchCriterion[]> {
    const separator =
      importProcessingContext.importSettings.additionalSettings
        ?.multiValueSeparator ?? ",";

    const criteria: EntityMatchCriterion[] = [];
    for (const { mapping, rawCell } of columns) {
      const config = normalizeEntityAdditional(mapping.additional);
      if (!config?.refField) {
        // column not usable as an identifier for this field
        continue;
      }

      const values: string[] = [];
      for (const rawValue of this.splitCellValues(
        rawCell,
        mapping,
        separator,
      )) {
        const value = await this.resolveColumnValue(
          rawValue,
          config.refField,
          config.valueMapping,
          context,
          importProcessingContext,
        );
        if (value !== undefined) {
          values.push(value);
        }
      }
      // the criterion is added even if no value could be resolved:
      // an empty list matches nothing, whereas dropping the criterion would
      // silently relax the condition and match unrelated records
      criteria.push({ refField: config.refField, values });
    }
    return criteria;
  }

  /**
   * Split a raw cell into the individual values to match, honoring the column's
   * enableSplitting flag.
   */
  private splitCellValues(
    rawCell: unknown,
    mapping: ColumnMapping,
    separator: string,
  ): unknown[] {
    if (rawCell === undefined || rawCell === null) {
      return [];
    }
    const enableSplitting = mapping.additional?.enableSplitting ?? true;
    return enableSplitting ? splitArrayValue(rawCell, separator) : [rawCell];
  }

  /**
   * Resolves the effective comparison value for a column,
   * applying any configured value mapping through the referenced field's datatype.
   *
   * @returns the normalized value to compare against, or undefined if the value
   *   mapping could not resolve the import value (which must match nothing)
   */
  private async resolveColumnValue(
    rawValue: any,
    refField: string,
    valueMapping: any | undefined,
    context: EntityFieldImportContext,
    importProcessingContext: ImportProcessingContext,
  ): Promise<string | undefined> {
    if (valueMapping === undefined) {
      return normalizeValue(rawValue);
    }

    const refFieldSchema = this.getRefFieldSchema(context.types, refField);
    const refDatatype = refFieldSchema
      ? this.schemaService.getDatatypeOrDefault(refFieldSchema.dataType)
      : null;

    if (!refDatatype) {
      return normalizeValue(rawValue);
    }

    const mappedValue = await refDatatype.importMapFunction(
      rawValue,
      refFieldSchema,
      valueMapping,
      importProcessingContext,
    );
    if (mappedValue === undefined || mappedValue === null) {
      // the import value could not be interpreted, so it cannot identify a record
      return undefined;
    }

    const dbFormat = refDatatype.transformToDatabaseFormat(
      mappedValue,
      refFieldSchema,
    );
    return normalizeValue(dbFormat);
  }

  /**
   * Load the candidates of every entity type the field may reference into the
   * context's cache, skipping types that are cached already.
   *
   * A field can allow referencing several entity types at once, in which case
   * `entityType` is an array rather than a single type name (as
   * `EditEntityComponent` handles for its own multi-type autocomplete, see
   * `edit-entity.component.ts`). Each type is loaded independently so that one
   * unresolvable type (e.g. a stale or removed registration) does not prevent
   * matching against the other, still-valid types.
   */
  private async loadImportMapEntities(
    entityType: string | string[],
    context: EntityFieldImportContext,
  ): Promise<void> {
    const missingTypes = asArray(entityType).filter(
      (type) => !context.hasEntitiesOfType(type),
    );

    await Promise.all(
      missingTypes.map(async (type) => {
        try {
          const entities = await this.entityMapper.loadType(type);
          context.setEntitiesOfType(
            type,
            entities.map((e) =>
              this.schemaService.transformEntityToDatabaseFormat(e),
            ),
          );
        } catch (error) {
          Logging.error("Error loading entities for import mapping:", error);
          // cache the empty result so the failing type is not retried for every
          // other field referencing it in this same import run
          context.setEntitiesOfType(type, []);
        }
      }),
    );
  }

  /**
   * Schema of the matching property, taken from whichever of the allowed types
   * declares it (they could in principle disagree on its datatype - first wins).
   */
  private getRefFieldSchema(
    types: string[],
    refField: string,
  ): EntitySchemaField | undefined {
    for (const type of types) {
      if (!this.entityRegistry.has(type)) continue;

      const refFieldSchema = this.entityRegistry
        .get(type)
        .schema?.get(refField);
      if (refFieldSchema) return refFieldSchema;
    }
    return undefined;
  }

  /**
   * An entity reference points to an independent record, which must not be affected
   * by anonymizing the record that holds the reference. The reference is removed instead,
   * as for any other datatype that does not support partial anonymization.
   *
   * (this overrides StringDatatype, which would otherwise retain the first character of the id)
   *
   * @param value
   * @param schemaField
   * @param parent
   */
  override async anonymize(
    value,
    schemaField: EntitySchemaField,
    parent,
  ): Promise<string | undefined> {
    return undefined;
  }

  private async loadRelatedEntitiesToString(
    value: string | string[],
    schemaField: EntitySchemaField,
  ): Promise<string[]> {
    if (!value) return [];

    const relatedEntitiesToStrings: string[] = [];

    const relatedEntitiesIds: string[] = Array.isArray(value) ? value : [value];
    for (const relatedEntityId of relatedEntitiesIds) {
      const entityType =
        Entity.extractTypeFromId(relatedEntityId) || schemaField.additional;
      const relatedEntity = await this.entityMapper
        .load(entityType, relatedEntityId)
        .catch(() => undefined);

      relatedEntitiesToStrings.push(relatedEntity?.toString() ?? "<not_found>");
    }

    return relatedEntitiesToStrings;
  }
}

/**
 * One matching condition derived from a mapped import column: candidates must
 * have `refField` equal to one of the (normalized) `values`.
 */
interface EntityMatchCriterion {
  refField: string;
  values: string[];
}

/**
 * Structure for the `additional` field of an entity reference ColumnMapping.
 * Can be a plain string (legacy) or an object with optional valueMapping config.
 */
export interface EntityAdditional {
  /** The property of the referenced entity to match against the import value. */
  refField: string;
  /** Optional: additional config for transforming the import value (passed to the sub-field's importMapFunction). */
  valueMapping?: any;
}

/**
 * Normalizes the `additional` config of an entity reference column mapping.
 * Accepts legacy string format or new object format.
 */
export function normalizeEntityAdditional(
  additional: string | EntityAdditional | any,
): EntityAdditional | undefined {
  if (!additional) {
    return undefined;
  }
  if (typeof additional === "string") {
    return { refField: additional };
  }
  return additional as EntityAdditional;
}

/**
 * Normalizes a value for comparison, converting it to a standardized string format.
 * Ensures both numbers and strings are treated consistently.
 *
 * @param val The value to normalize.
 * @returns The normalized value as a string.
 */
function normalizeValue(val: any): string {
  if (val == null) {
    return "";
  }
  return String(val).trim().toLowerCase(); // Convert everything to string and trim spaces
}

/**
 * Manage cache access to the current import processing context.
 */
class EntityFieldImportContext {
  constructor(
    private globalContext: ImportProcessingContext,
    private schemaField: EntitySchemaField,
  ) {}

  /**
   * All entity types this field may reference
   * (an array when the field allows several target types).
   */
  get types(): string[] {
    return asArray(this.schemaField.additional);
  }

  /**
   * Candidates (in database format for easier comparison!) of every type this
   * field allows, merged.
   *
   * Cached per entity type rather than per field, so a type loaded for one
   * field is reused by every other field referencing it in the same import
   * run - including fields that allow only a subset of these types.
   */
  get entities(): any[] {
    return this.types.flatMap((type) => this.entitiesOfType(type) ?? []);
  }

  hasEntitiesOfType(type: string): boolean {
    return !!this.entitiesOfType(type);
  }

  setEntitiesOfType(type: string, entities: any[]) {
    this.globalContext[`entities_${type}`] = entities;
  }

  private entitiesOfType(type: string): any[] | undefined {
    return this.globalContext[`entities_${type}`];
  }
}

results matching ""

    No results matching ""