Query

Queries are the most efficient way to retrieve items from DynamoDB, which is why it is recommend to be very thoughtful when modeling your indexes.

Example Setup

Table Definition

{
  "TableName": "electro",
  "KeySchema": [
    {
      "AttributeName": "pk",
      "KeyType": "HASH"
    },
    {
      "AttributeName": "sk",
      "KeyType": "RANGE"
    }
  ],
  "AttributeDefinitions": [
    {
      "AttributeName": "pk",
      "AttributeType": "S"
    },
    {
      "AttributeName": "sk",
      "AttributeType": "S"
    },
    {
      "AttributeName": "gsi1pk",
      "AttributeType": "S"
    },
    {
      "AttributeName": "gsi1sk",
      "AttributeType": "S"
    }
  ],
  "GlobalSecondaryIndexes": [
    {
      "IndexName": "gsi1pk-gsi1sk-index",
      "KeySchema": [
        {
          "AttributeName": "gsi1pk",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "gsi1sk",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    },
    {
      "IndexName": "gsi2pk-gsi2sk-index",
      "KeySchema": [
        {
          "AttributeName": "gsi2pk",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "gsi2sk",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    }
  ],
  "BillingMode": "PAY_PER_REQUEST"
}

Example Entity

import DynamoDB from "aws-sdk/clients/dynamodb";
import { Entity } from "electrodb";

const client = new DynamoDB.DocumentClient();

const table = "electro";

const StoreLocations = new Entity(
  {
    model: {
      service: "MallStoreDirectory",
      entity: "MallStore",
      version: "1",
    },
    attributes: {
      cityId: {
        type: "string",
        required: true,
      },
      mallId: {
        type: "string",
        required: true,
      },
      storeId: {
        type: "string",
        required: true,
      },
      buildingId: {
        type: "string",
        required: true,
      },
      unitId: {
        type: "string",
        required: true,
      },
      category: {
        type: [
          "spite store",
          "food/coffee",
          "food/meal",
          "clothing",
          "electronics",
          "department",
          "misc",
        ],
        required: true,
      },
      leaseEndDate: {
        type: "string",
        required: true,
      },
      rent: {
        type: "string",
        required: true,
        validate: /^(\d+\.\d{2})$/,
      },
      discount: {
        type: "string",
        required: false,
        default: "0.00",
        validate: /^(\d+\.\d{2})$/,
      },
      tenants: {
        type: "set",
        items: "string",
      },
      warnings: {
        type: "number",
        default: 0,
      },
      deposit: {
        type: "number",
      },
      contact: {
        type: "set",
        items: "string",
      },
      rentalAgreement: {
        type: "list",
        items: {
          type: "map",
          properties: {
            type: {
              type: "string",
            },
            detail: {
              type: "string",
            },
          },
        },
      },
      petFee: {
        type: "number",
      },
      fees: {
        type: "number",
      },
      tags: {
        type: "set",
        items: "string",
      },
    },
    indexes: {
      stores: {
        pk: {
          field: "pk",
          composite: ["cityId", "mallId"],
        },
        sk: {
          field: "sk",
          composite: ["buildingId", "storeId"],
        },
      },
      units: {
        index: "gsi1pk-gsi1sk-index",
        pk: {
          field: "gsi1pk",
          composite: ["mallId"],
        },
        sk: {
          field: "gsi1sk",
          composite: ["buildingId", "unitId"],
        },
      },
      leases: {
        index: "gsi2pk-gsi2sk-index",
        pk: {
          field: "gsi2pk",
          composite: ["storeId"],
        },
        sk: {
          field: "gsi2sk",
          composite: ["leaseEndDate"],
        },
      },
    },
  },
  { table, client },
);

(Example code on this page that references the entity StoreLocations uses the following Entity and Table Definition found below)

Query Items

ElectroDB queries use DynamoDB’s query method to retrieve items based on your table’s indexes.

Forming a composite Partition Key and Sort Key is a critical step in planning Access Patterns in DynamoDB. When planning composite keys, it is crucial to consider the order in which they are composed. As of the time of writing this documentation, DynamoDB has the following constraints that should be taken into account when planning your Access Patterns:

You must always supply the Partition Key in full for all queries to DynamoDB.
You currently have the following operators available to narrow results via your Sort Key
To act on single record, you will need to know the full Partition Key and Sort Key for that record.

Providing composite attributes to queries

While ElectroDB does abstract out the formatting of your index keys, you will still need to provide ElectroDB with the values necessary to build your keys. The following examples use the StoreLocations Entity defined in the Example Setup section above.

Good: Includes at least the PK

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
  })
  .go();

Good: Includes at least the PK, and the first SK attribute

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
    buildingId: "f34",
  })
  .go();

Good: Includes at least the PK, and all the SK attributes

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
    storeId: "LatteLarrys",
    buildingId: "f34",
  })
  .go();

Bad: No PK composite attributes specified

With DynamoDB, queries require at least a Partition Key is provided. Without specifying the partition key composite attributes (cityId and mallId), ElectroDB is unable to build a key to supply to DynamoDB. This example will throw.

await StoreLocations.query.stores().go();

Bad: Not All PK Composite Attributes included

The ElectroDB cannot form a Partition Key without a cityId, this example will throw.

await StoreLocations.query
  .stores({
    mallId: "EastPointe",
  })
  .go();

Bad: Composite Attributes not included in order

This example will _not throw, but will it ignore unitId because storeId was not supplied and because ElectroDB cannot logically build a valid sort key string without the all values being provided left to right.

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
    unitId: "B24",
  })
  .go();

Using composite attributes to make hierarchical keys

Carefully considering the sequential order Composite Attributes are defined will allow ElectroDB to express hierarchical relationships and unlock more available Access Patterns for your application.

For a comprehensive and interactive guide to build queries please visit this runkit: https://runkit.com/tywalch/electrodb-building-queries.

Access Pattern Queries

When you define your indexes in your model, you are defining the Access Patterns of your entity. The composite attributes you choose, and their order, ultimately define the finite set of index queries that can be made. The more you can leverage these index queries the better from both a cost and performance perspective.

Unlike Partition Keys, Sort Keys can be partially provided. We can leverage this to multiply our available access patterns and use the Sort Key Operations: begins, between, lt, lte, gt, and gte. These queries are more performant and cost-effective than filters. The costs associated with DynamoDB directly correlate to how effectively you leverage Sort Key Operations.

For a comprehensive and interactive guide to build queries please visit this runkit: https://runkit.com/tywalch/electrodb-building-queries.

Partition Key Composite Attributes

All queries require (at minimum) the Composite Attributes included in its defined Partition Key. Composite Attributes you define on the Sort Key can be partially supplied, but must be supplied in the order they are defined.

Composite Attributes must be supplied in the order they are composed when invoking the Access Pattern*. This is because composite attributes are used to form a concatenated key string, and if attributes supplied out of order, it is not possible to fill the gaps in that concatenation.

Sort Key Operations

operator	use case
`begins`	Keys starting with a particular set of characters.
`between`	Keys between a specified range.
`gt`	Keys greater than some value
`gte`	Keys greater than or equal to some value
`lt`	Keys less than some value
`lte`	Keys less than or equal to some value

Begins With Queries

One important consideration when using Sort Key Operations to make is when to use and not to use “begins”.

It is possible to supply partially supply Sort Key composite attributes. Sort Key attributes must be provided in the order they are defined, but it’s possible to provide only a subset of the Sort Key Composite Attributes to ElectroDB. By default, when you supply a partial Sort Key in the Access Pattern method, ElectroDB will create a beginsWith query. The difference between that and using .begins() is that, with a .begins() query, ElectroDB will not post-pend the next composite attribute’s label onto the query.

The difference is nuanced and makes better sense with an example, but the rule of thumb is that data passed to the Access Pattern method should represent values you know strictly equal the value you want.

The following examples will use the following Access Pattern definition for units:

{
  "units": {
    "index": "gsi1pk-gsi1sk-index",
    "pk": {
      "field": "gsi1pk",
      "composite": ["mallId"]
    },
    "sk": {
      "field": "gsi1sk",
      "composite": ["buildingId", "unitId"]
    }
  }
}

The names you have given to your indexes on your entity model/schema express themselves as “Access Pattern” methods on your Entity’s query object:

// Example #1, access pattern `units`
StoreLocations.query.units({ mallId, buildingId }).go();
// -----------------------^^^^^^^^^^^^^^^^^^^^^^

Data passed to the Access Pattern method is considered to be full, known, data. In the above example, we are saying we know the mallId, buildingId and unitId.

Alternatively, if you only know the start of a piece of data, use .begins():

// Example #2
StoreLocations.query.units({ mallId }).begins({ buildingId }).go();
// ---------------------------------^^^^^^^^^^^^^^^^^^^^^

Data passed to the .begins() method is considered to be partial data. In the second example, we are saying we know the mallId and buildingId, but only know the beginning of unitId.

For the above queries we see two different sort keys:

"$mallstore_1#buildingid_f34#unitid_"
"$mallstore_1#buildingid_f34"

The first example shows how ElectroDB post-pends the label of the next composite attribute (unitId) on the Sort Key to ensure that buildings such as "f340" are not included in the query. This is useful to prevent common issues with overloaded sort keys like accidental over-querying.

The second example allows you to make queries that do include buildings such as "f340" or "f3409" or "f340356346".

For these reasons it is important to consider that attributes passed to the Access Pattern method are considered to be full, known, data.

Examples

The following examples use the Entity and table configuration defined in the section Example Setup above.

Lease Agreements by StoreId

await StoreLocations.query.leases({ storeId: "LatteLarrys" }).go();

Lease Agreement by StoreId for March 22nd 2020

await StoreLocations.query
  .leases({
    storeId: "LatteLarrys",
    leaseEndDate: "2020-03-22",
  })
  .go();

Lease agreements by StoreId for 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .begins({ leaseEndDate: "2020" })
  .go();

Lease Agreements by StoreId after March 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .gt({ leaseEndDate: "2020-03" })
  .go();

Lease Agreements by StoreId after, and including, March 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .gte({ leaseEndDate: "2020-03" })
  .go();

Lease Agreements by StoreId before 2021

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .lt({ leaseEndDate: "2021-01" })
  .go();

Lease Agreements by StoreId before February 2021

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .lte({ leaseEndDate: "2021-02" })
  .go();

Lease Agreements by StoreId between 2010 and 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .between({ leaseEndDate: "2010" }, { leaseEndDate: "2020" })
  .go();

Lease Agreements by StoreId after, and including, 2010 in the city of Atlanta and category containing food

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .gte({ leaseEndDate: "2010" })
  .where(
    (attr, op) => `
      ${op.eq(attr.cityId, "Atlanta1")} AND ${op.contains(
        attr.category,
        "food",
      )}
  `,
  )
  .go();

Execution Options

Query options can be added the .params() and .go() to change query behavior or add customer parameters to a query.

By default, ElectroDB enables you to work with records as the names and properties defined in the model. Additionally, it removes the need to deal directly with the docClient parameters which can be complex for a team without as much experience with DynamoDB. The Query Options object can be passed to both the .params() and .go() methods when building you query. Below are the options available:

{
  params?: object;
  table?: string;
  data?: 'raw' | 'includeKeys' | 'attributes';
  pager?: 'raw' | 'cursor';
  originalErr?: boolean;
  response?: "default" | "none" | "all_old" | "updated_old" | "all_new" | "updated_new";
  ignoreOwnership?: boolean;
  limit?: number;
  count?: number;
  pages?: number | 'all';
  logger?: (event) => void;
  listeners Array<(event) => void>;
  attributes?: string[];
  order?: 'asc' | 'desc';
};

Query Execution Options

params

Default Value: {} Properties added to this object will be merged onto the params sent to the document client. Any conflicts with ElectroDB will favor the params specified here. Consider this an escape hatch for options not yet supported by ElectroDB.

table

Default Value: (from constructor) Use a different table than the one defined in Service Options and/or Entity Options.

attributes

Default Value: (all attributes) The attributes query option allows you to specify ProjectionExpression Attributes for your get or query operation.

data

Default Value: "attributes" Accepts the values 'raw', 'includeKeys', 'attributes' or undefined. Use 'raw' to return query results as they were returned by the docClient. Use 'includeKeys' to include item partition and sort key values in your return object. By default, ElectroDB does not return partition, sort, or global keys in its response.

originalErr

Default Value: false By default, ElectroDB alters the stacktrace of any exceptions thrown by the DynamoDB client to give better visibility to the developer. Set this value equal to true to turn off this functionality and return the error unchanged.

response

Default Value: "default" Used as a convenience for applying the DynamoDB parameter ReturnValues. The options here are the same as the parameter values for the DocumentClient except lowercase. The "none" option will cause the method to return null and will bypass ElectroDB’s response formatting — useful if formatting performance is a concern.

ignoreOwnership

Default Value: false By default, ElectroDB interrogates items returned from a query for the presence of matching entity “identifiers”. This helps to ensure other entities, or other versions of an entity, are filtered from your results. If you are using ElectroDB with an existing table/dataset you can turn off this feature by setting this property to true.

limit

Default Value: none Used a convenience wrapper for the DynamoDB parameter Limit [read more]. When limit is paired option pages:"all", ElectroDB will paginate DynamoDB until the limit is at least reached or all items for that query have been returned; this means you may receive more items than your specified limit but never less.

Note: If your use case requires returning a specific number of items at a time, the count option is the better option.

count

Default Value: none A target for the number of items to return from DynamoDB. If this option is passed, Queries on entities will paginate DynamoDB until the items found match the count is reached or all items for that query have been returned. It is important to understand that this option may result in ElectroDB making multiple calls to DynamoDB to satisfy the count. For this reason, you should also consider using the pages option to limit the number of requests (or “pages”) made to DynamoDB.

pages

Default Value: 1 How many DynamoDB pages should a query iterate through before stopping. To have ElectroDB automatically paginate through all results, pass the string value 'all'.

order

Default Value: ‘asc’ Convenience option for ScanIndexForward, to the change the order of queries based on your index’s Sort Key — valid options include ‘asc’ and ‘desc’. [read more]

listeners

Default Value: [] An array of callbacks that are invoked when internal ElectroDB events occur.

logger

Default Value: none A convenience option for a single event listener that semantically can be used for logging.