Query
Queries are the most efficient way to retrieve items from DynamoDB, which is why it is recommended that you be very thoughtful when modeling your indexes.
Example Setup
Table Definition
{ "TableName": "electro", "KeySchema": [ { "AttributeName": "pk", "KeyType": "HASH" }, { "AttributeName": "sk", "KeyType": "RANGE" } ], "AttributeDefinitions": [ { "AttributeName": "pk", "AttributeType": "S" }, { "AttributeName": "sk", "AttributeType": "S" }, { "AttributeName": "gsi1pk", "AttributeType": "S" }, { "AttributeName": "gsi1sk", "AttributeType": "S" } ], "GlobalSecondaryIndexes": [ { "IndexName": "gsi1pk-gsi1sk-index", "KeySchema": [ { "AttributeName": "gsi1pk", "KeyType": "HASH" }, { "AttributeName": "gsi1sk", "KeyType": "RANGE" } ], "Projection": { "ProjectionType": "ALL" } }, { "IndexName": "gsi2pk-gsi2sk-index", "KeySchema": [ { "AttributeName": "gsi2pk", "KeyType": "HASH" }, { "AttributeName": "gsi2sk", "KeyType": "RANGE" } ], "Projection": { "ProjectionType": "ALL" } } ], "BillingMode": "PAY_PER_REQUEST" }
(Example code on this page that references the entityExample Entity
import DynamoDB from "aws-sdk/clients/dynamodb"; import { Entity } from "electrodb"; const client = new DynamoDB.DocumentClient(); const table = "electro"; const StoreLocations = new Entity( { model: { service: "MallStoreDirectory", entity: "MallStore", version: "1", }, attributes: { cityId: { type: "string", required: true, }, mallId: { type: "string", required: true, }, storeId: { type: "string", required: true, }, buildingId: { type: "string", required: true, }, unitId: { type: "string", required: true, }, category: { type: [ "spite store", "food/coffee", "food/meal", "clothing", "electronics", "department", "misc", ], required: true, }, leaseEndDate: { type: "string", required: true, }, rent: { type: "string", required: true, validate: /^(\d+\.\d{2})$/, }, discount: { type: "string", required: false, default: "0.00", validate: /^(\d+\.\d{2})$/, }, tenants: { type: "set", items: "string", }, warnings: { type: "number", default: 0, }, deposit: { type: "number", }, contact: { type: "set", items: "string", }, rentalAgreement: { type: "list", items: { type: "map", properties: { type: { type: "string", }, detail: { type: "string", }, }, }, }, petFee: { type: "number", }, fees: { type: "number", }, tags: { type: "set", items: "string", }, }, indexes: { stores: { pk: { field: "pk", composite: ["cityId", "mallId"], }, sk: { field: "sk", composite: ["buildingId", "storeId"], }, }, units: { index: "gsi1pk-gsi1sk-index", pk: { field: "gsi1pk", composite: ["mallId"], }, sk: { field: "gsi1sk", composite: ["buildingId", "unitId"], }, }, leases: { index: "gsi2pk-gsi2sk-index", pk: { field: "gsi2pk", composite: ["storeId"], }, sk: { field: "gsi2sk", composite: ["leaseEndDate"], }, }, }, }, { table, client }, );
StoreLocations
uses the following Entity and Table Definition found below)
Query Items
ElectroDB queries use DynamoDB’s query
method to retrieve items based on your table’s indexes.
Designing a composite Partition Key and Sort Key is a critical step in planning Access Patterns in DynamoDB. When planning composite keys, it is crucial to consider the order in which they are composed. As of the time of writing this documentation, DynamoDB has the following constraints that should be taken into account when planning your Access Patterns:
- You must always supply the Partition Key in full for all queries to DynamoDB.
- You currently have the following operators available to narrow results via your Sort Key
- To act on a single record, you must know the full Partition Key and Sort Key for that record.
Providing composite attributes to queries
While ElectroDB does abstract out the formatting of your index keys, you will still need to provide ElectroDB with the values necessary to build your keys. The following examples use the StoreLocations
Entity defined in the Example Setup section above.
Good: Includes at least the PK
await StoreLocations.query
.stores({
cityId: "Atlanta1",
mallId: "EastPointe",
})
.go();
Good: Includes at least the PK, and the first SK attribute
await StoreLocations.query
.stores({
cityId: "Atlanta1",
mallId: "EastPointe",
buildingId: "f34",
})
.go();
Good: Includes at least the PK, and all the SK attributes
await StoreLocations.query
.stores({
cityId: "Atlanta1",
mallId: "EastPointe",
storeId: "LatteLarrys",
buildingId: "f34",
})
.go();
Bad: No PK composite attributes specified
With DynamoDB, queries require at least a Partition Key is provided. Without specifying the partition key composite attributes (cityId
and mallId
), ElectroDB is unable to build a key to supply to DynamoDB. This example will throw.
await StoreLocations.query.stores().go();
Bad: Not All PK Composite Attributes included
The ElectroDB cannot form a Partition Key without a cityId
, this example will throw.
await StoreLocations.query
.stores({
mallId: "EastPointe",
})
.go();
Bad: Composite Attributes not included in order
This example will not throw, but will it ignore unitId
because storeId
was not supplied ElectroDB cannot logically build a valid sort key string without the all values being provided left to right.
await StoreLocations.query
.stores({
cityId: "Atlanta1",
mallId: "EastPointe",
unitId: "B24",
})
.go();
Using composite attributes to make hierarchical keys
Carefully considering the sequential order Composite Attributes are defined will allow ElectroDB to express hierarchical relationships and unlock more available Access Patterns for your application.
For a comprehensive and interactive guide to build queries please visit this runkit: https://runkit.com/tywalch/electrodb-building-queries.
Access Pattern Queries
When you define your indexes in your model, you are defining the Access Patterns of your entity. The composite attributes you choose, and their order, ultimately define the finite set of index queries that can be made. The more you can leverage these index queries the better from both a cost and performance perspective.
Unlike Partition Keys, Sort Keys can be partially provided. We can leverage this to multiply our available access patterns and use the Sort Key Operations: begins
, between
, lt
, lte
, gt
, and gte
. These queries are more performant and cost-effective than filters. The costs associated with DynamoDB directly correlate to how effectively you leverage Sort Key Operations.
For a comprehensive and interactive guide to build queries please visit this runkit: https://runkit.com/tywalch/electrodb-building-queries.
Partition Key Composite Attributes
All queries require (at minimum) the Composite Attributes included in its defined Partition Key. Composite Attributes you define on the Sort Key can be partially supplied, but must be supplied in the order they are defined.
Composite Attributes must be supplied in the order they are composed when invoking the Access Pattern*. This is because composite attributes are used to form a concatenated key string, and if attributes are supplied out of order, it is not possible to fill the gaps in that concatenation.
Sort Key Operations
operator | use case |
---|---|
begins | Keys starting with a particular set of characters. |
between | Keys between a specified range. |
gt | Keys greater than some value |
gte | Keys greater than or equal to some value |
lt | Keys less than some value |
lte | Keys less than or equal to some value |
Begins With Queries
One important consideration when using Sort Key Operations is when to use and not to use “begins”.
It is possible to supply partially supply Sort Key composite attributes. Sort Key attributes must be provided in the order they are defined, but it’s possible to provide only a subset of the Sort Key Composite Attributes to ElectroDB. By default, when you supply a partial Sort Key in the Access Pattern method, ElectroDB will create a beginsWith
query. The difference between that and using .begins()
is that, with a .begins()
query, ElectroDB will not post-pend the next composite attribute’s label onto the query.
The difference is nuanced and makes better sense with an example, but the rule of thumb is that data passed to the Access Pattern method should represent values you know strictly equal the value you want.
The following examples will use the following Access Pattern definition for units
:
{
"units": {
"index": "gsi1pk-gsi1sk-index",
"pk": {
"field": "gsi1pk",
"composite": ["mallId"]
},
"sk": {
"field": "gsi1sk",
"composite": ["buildingId", "unitId"]
}
}
}
The names you have given to your indexes on your entity model/schema express themselves as “Access Pattern” methods on your Entity’s query
object:
// Example #1, access pattern `units`
StoreLocations.query.units({ mallId, buildingId }).go();
// -----------------------^^^^^^^^^^^^^^^^^^^^^^
Data passed to the Access Pattern method is considered to be full, known, data. In the above example, we are saying we know the mallId
, buildingId
and unitId
.
Alternatively, if you only know the start of a piece of data, use .begins():
// Example #2
StoreLocations.query.units({ mallId }).begins({ buildingId }).go();
// ---------------------------------^^^^^^^^^^^^^^^^^^^^^
Data passed to the .begins() method is considered to be partial data. In the second example, we are saying we know the mallId
and buildingId
, but only know the beginning of unitId
.
For the above queries we see two different sort keys:
"$mallstore_1#buildingid_f34#unitid_"
"$mallstore_1#buildingid_f34"
The first example shows how ElectroDB post-pends the label of the next composite attribute (unitId
) on the Sort Key to ensure that buildings such as "f340"
are not included in the query. This is useful to prevent common issues with overloaded sort keys like accidental over-querying.
The second example allows you to make queries that do include buildings such as "f340"
or "f3409"
or "f340356346"
.
For these reasons it is important to consider that attributes passed to the Access Pattern method are considered to be full, known, data.
Examples
The following examples use the Entity and table configuration defined in the section Example Setup above.
Lease Agreements by StoreId
await StoreLocations.query.leases({ storeId: "LatteLarrys" }).go();
Lease Agreement by StoreId for March 22nd 2020
await StoreLocations.query
.leases({
storeId: "LatteLarrys",
leaseEndDate: "2020-03-22",
})
.go();
Lease agreements by StoreId for 2020
await StoreLocations.query
.leases({ storeId: "LatteLarrys" })
.begins({ leaseEndDate: "2020-00-00" })
.go();
Lease Agreements by StoreId after March 2020
await StoreLocations.query
.leases({ storeId: "LatteLarrys" })
.gt({ leaseEndDate: "2020-04-00" })
.go();
Lease Agreements by StoreId after, and including, March 2020
await StoreLocations.query
.leases({ storeId: "LatteLarrys" })
.gte({ leaseEndDate: "2020-03-00" })
.go();
Lease Agreements by StoreId before 2021
await StoreLocations.query
.leases({ storeId: "LatteLarrys" })
.lt({ leaseEndDate: "2021-00-00" })
.go();
Lease Agreements by StoreId before February 2021
await StoreLocations.query
.leases({ storeId: "LatteLarrys" })
.lte({ leaseEndDate: "2021-02-00" })
.go();
Lease Agreements by StoreId between 2010 and 2020
await StoreLocations.query
.leases({ storeId: "LatteLarrys" })
.between({ leaseEndDate: "2010-00-00" }, { leaseEndDate: "2020-99-99" })
.go();
Lease Agreements by StoreId after, and including, 2010 in the city of Atlanta and category containing food
await StoreLocations.query
.leases({ storeId: "LatteLarrys" })
.gte({ leaseEndDate: "2010-00-00" })
.where(
(attr, op) => `
${op.eq(attr.cityId, "Atlanta1")} AND ${op.contains(
attr.category,
"food",
)}
`,
)
.go();
Comparison Queries
When performing comparison queries (e.g. gt
, gte
, lte
, lt
, between
) it is often necessary to add additional filters. Understanding how a comparison query will impact the items returned can be challenging due to the unintuitive nature of sorting and multi-composite attribute Sort Keys.
By default, ElectroDB builds comparison queries without applying attribute filters; the execution option { compare: "keys" }
and not specifying compare
are functionally equivalent. In this way, the query’s comparison applies to the item’s “keys”. This approach gives you complete control of your indexes, though your query may return unexpected data if you are not careful with item sorting.
When using { compare: "attributes" }
, the compare
option instructs ElectroDB to apply attribute filters on each Sort Key composite attribute provided. In this way, the query comparison applies to the item’s “attributes.”
The option { compare: "v2" }
offers backwards compatibility with the ElectroDB v2
API. This option exists to assist in migration, should be avoided for greenfield projects, and will be removed in the near future.
Execution Options
Query options can be added the .params()
and .go()
to change query behavior or add customer parameters to a query.
By default, ElectroDB enables you to work with records as the names and properties defined in the model. Additionally, it removes the need to deal directly with the docClient parameters which can be complex for a team without as much experience with DynamoDB. The Query Options object can be passed to both the .params()
and .go()
methods when building you query. Below are the options available:
{
params?: object;
table?: string;
data?: 'raw' | 'includeKeys' | 'attributes';
pager?: 'raw' | 'cursor';
originalErr?: boolean;
response?: "default" | "none" | "all_old" | "updated_old" | "all_new" | "updated_new";
ignoreOwnership?: boolean;
limit?: number;
count?: number;
pages?: number | 'all';
logger?: (event) => void;
listeners Array<(event) => void>;
attributes?: string[];
order?: 'asc' | 'desc';
compare?: 'attributes' | 'keys' | 'v2';
};
Query Execution Options
params
Default Value: {}
Properties added to this object will be merged onto the params sent to the document client. Any conflicts with ElectroDB will favor the params specified here. Consider this an escape hatch for options not yet supported by ElectroDB.
table
Default Value: (from constructor) Use a different table than the one defined in Service Options and/or Entity Options.
attributes
Default Value: (all attributes)
The attributes
query option allows you to specify ProjectionExpression Attributes for your get
or query
operation.
data
Default Value: "attributes"
Accepts the values 'raw'
, 'includeKeys'
, 'attributes'
or undefined
. Use 'raw'
to return query results as they were returned by the docClient. Use 'includeKeys'
to include item partition and sort key values in your return object. By default, ElectroDB does not return partition, sort, or global keys in its response.
originalErr
Default Value: false
By default, ElectroDB alters the stacktrace of any exceptions thrown by the DynamoDB client to give better visibility to the developer. Set this value equal to true
to turn off this functionality and return the error unchanged.
response
Default Value: "default"
Used as a convenience for applying the DynamoDB parameter ReturnValues
. The options here are the same as the parameter values for the DocumentClient except lowercase. The "none"
option will cause the method to return null and will bypass ElectroDB’s response formatting — useful if formatting performance is a concern.
ignoreOwnership
Default Value: false
By default, ElectroDB interrogates items returned from a query for the presence of matching entity “identifiers”. This helps to ensure other entities, or other versions of an entity, are filtered from your results. If you are using ElectroDB with an existing table/dataset you can turn off this feature by setting this property to true
.
limit
Default Value: none
Used a convenience wrapper for the DynamoDB parameter Limit
[read more].
count
Default Value: none
A target for the number of items to return from DynamoDB. If this option is passed, Queries on entities will paginate DynamoDB until the items found match the count is reached or all items for that query have been returned. It is important to understand that this option may result in ElectroDB making multiple calls to DynamoDB to satisfy the count. For this reason, you should also consider using the pages
option to limit the number of requests (or “pages”) made to DynamoDB.
pages
Default Value: 1
How many DynamoDB pages should a query iterate through before stopping. To have ElectroDB automatically paginate through all results, pass the string value 'all'
.
order
Default Value: ‘asc’
Convenience option for ScanIndexForward
, to the change the order of queries based on your index’s Sort Key — valid options include ‘asc’ and ‘desc’. [read more]
listeners
Default Value: []
An array of callbacks that are invoked when internal ElectroDB events occur.
logger
Default Value: none A convenience option for a single event listener that semantically can be used for logging.
compare
Default Value: ‘keys’
When performing comparison queries (e.g. gt
, gte
, lte
, lt
, between
) it is often necessary to add additional filters. Understanding how a comparison query will impact the items returned can be challenging due to the unintuitive nature of sorting and multi-composite attribute Sort Keys.
By default (comparison: "keys"
), ElectroDB builds comparison queries without applying attribute filters. In this way, the query’s comparison applies to the item’s “keys”. This approach gives you complete control of your indexes, though your query may return unexpected data if you are not careful with item sorting.
When set to attributes
, the comparison
option instructs ElectroDB to apply attribute filters on each Sort Key composite attribute provided. In this way, the query comparison applies to the item’s “attributes.”