Utility Scripts for Firebase ProjectsUtility Scripts for Firebase Projects
๐Ÿ”จ
Utility Scripts for Firebase Projects
Date: Nov 21, 2020
For projects that have something to do with Firebase products (eg. Firebase Authentication, Firestore, etc.), I keep a directory scripts in project root. This directory contains scripts that I use to interact with Firebase from time to time. The directory must have access to firebase-admin package because the scripts depend on it to work.
If you want to use the scripts in your project, you can copy the script, create new file in your project where your feel convenient, then paste the script in that new file. Or you can clone the github repo: https://github.com/wzulfikar/firebase-utility-scripts.
ย 

๐Ÿงฉ Conventions

Each script will have the following convention:
  • All scripts should work with firebase-admin v9 (it was firebase-admin@^9.4.1 at the time of this writing). Other version of firebase-admin is not tested.
  • All scripts require SERVICE_ACCOUNT_FILE environment variable to determine the location of service account file. Consider using direnv to automate the configuration of such environment variables. You don't have to set up direnv to make the script works, but it'll make your life easier. For example, because I have direnv set up in my machine, the environment variables for my project will be automatically configured by having this .envrc file in my project directory:
    • # File: .envrc
      
      export SERVICE_ACCOUNT_FILE=${PWD}/acme-app-1234-firebase-adminsdk-d8yne-d29f9ed054.json
  • By default, scripts will use project_id from specified SERVICE_ACCOUNT_FILE and use it to form Firestore database url in this format: https://{project_id}.firebaseio.com. This could be your production endpoint, so please be sure that you know what you're doing. If you want to override the database url, create an environment variable FIRESTORE_DATABASE_URL and the script will use the value from there. Specifying custom database url is useful when you are testing locally and you wan to connect to your local firebase emulator. For example, with my direnv workflow, my .envrc file will look like this:
    • # File: .envrc
      
      export SERVICE_ACCOUNT_FILE=${PWD}/acme-app-1234-firebase-adminsdk-d8yne-d29f9ed054.json
      export FIRESTORE_DATABASE_URL=http://localhost:8080
  • All scripts accepts -h or --help to display the usage info
  • When a script that requires an argument is ran without arguments, it will display the usage info. For example, if the script is listDocument.js, running node listDocument.js is the same as running node listDocument.js --help
ย 

๐Ÿงณ The Scripts

countDocuments.js
โ„น๏ธ Usage:
Get number of documents in given collection.
๐Ÿญ Sample output:
notion imagenotion image
๐Ÿ–‹ The script:
// Ignore first, second items of argv (node bin path and script path)
const [, , collectionName] = process.argv;

if (!collectionName || collectionName == "-h" || collectionName == "--help") {
  console.log("Usage\t: node countDocuments.js <collectionName>");
  console.log("Example\t: node countDocuments.js users");
  process.exit(0);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

admin
  .firestore()
  .collection(collectionName)
  .get()
  .then((snapshot) => {
    if (snapshot.empty) {
      console.log("[INFO] collection did not contain any documents yet");
      return;
    }
    console.log(`[INFO] size of ${collectionName}:`, snapshot.size);
  });
ย 
findByField.js
โ„น๏ธ Usage:
Find rows in Firestore database based on specified field value.
๐Ÿญ Sample output:
(No sample output at this moment)
๐Ÿ–‹ The script:
// Ignore first, second items of argv (node bin path and script path)
const [, , collectionName, fieldName, operator, findValue] = process.argv;

if (!collectionName || collectionName == "-h" || collectionName == "--help") {
  console.log(
    "Retrieve rows in Firestore database based on specified field value."
  );
  console.log(
    "Usage\t: node findByField.js <collectionName> <columnName> <operator> <findValue>"
  );
  console.log(
    "Example\t: node findByField.js users email '==' 'john@example.com'"
  );
  process.exit(0);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

const firebaseAdmin = admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

const firestore = firebaseAdmin.firestore();

firestore
  .collection(collectionName)
  .where(fieldName, operator, findValue)
  .get()
  .then((snapshot) => {
    if (snapshot.empty) {
      console.log("[INFO] documents not found");
      return;
    }

    snapshot.forEach((doc) => {
      console.log(doc.data());
    });
  });
findFirebaseUser.js
โ„น๏ธ Usage:
Find user in Firebase Authentication by email.
๐Ÿญ Sample output:
notion imagenotion image
๐Ÿ–‹ The script:
// Ignore first, second items of argv (node bin path and script path)
const [, , email] = process.argv;

if (!email || email == "-h" || email == "--help") {
  console.log("Find user in Firebase Authentication by email.");
  console.log("Usage\t: node findFirebaseUser.js <email>");
  console.log("Example\t: node findFirebaseUser.js john@example.com");
  process.exit(0);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

admin
  .auth()
  .getUserByEmail(email)
  .then((user) => {
    console.log(user.toJSON());
    process.exit();
  })
  .catch((err) => console.log("[ERROR] could not get user:", err.message));
ย 
importJson.js
โ„น๏ธ Usage:
Import data from json file to Firestore.
๐Ÿญ Sample output:
(No sample output for the moment).
๐Ÿ–‹ The script:
// Ignore first, second items of argv (node bin path and script path)
const [, , jsonFile, collectionName, documentId] = process.argv;

if (!collectionName || collectionName == "-h" || collectionName == "--help") {
  console.log("Import data from json file to Firestore.");
  console.log(
    "Usage\t: node importJson.js <collectionName> <jsonFile> <documentId=null>"
  );
  console.log("Example\t: node importJson.js users users.json id");
  process.exit(0);
}

function sleep(ms) {
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve("resolved");
    }, ms);
  });
}

async function commitBatches(batches) {
  for (let i = 0; i < batches.length; i++) {
    await sleep(1000);
    await batches[i].commit();
    console.log(`โœ” batch ${i + 1} of ${batches.length}`);
  }
}

const fs = require("fs");
const JSONStream = require("JSONStream");
const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

const firestore = admin.firestore();

let rowCount = 0;

let counter = 0;
let commitCounter = 0;
const batches = [];
batches[commitCounter] = firestore.batch();

const fieldTypes = {
  timestamp: [
    "created_at",
    "updated_at",
    "deleted_at",
    "submitted_at",
    "createdAt",
    "updatedAt",
    "deletedAt",
    "submittedAt",
  ],
};

fs.createReadStream(jsonFile)
  .pipe(JSONStream.parse("*"))
  .on("data", (data) => {
    rowCount++;

    if (counter < 499) {
      if (documentId && !data[documentId]) {
        console.log(
          `[ERROR] invalid document id at row ${rowCount}:`,
          data[documentId],
          data
        );
        process.exit();
      }

      let ref = null;
      if (documentId) {
        ref = firestore.collection(collectionName).doc(data[documentId]);
      } else {
        ref = firestore.collection(collectionName).doc();
      }

      if (documentId) {
        delete data[documentId];
      }

      // Uncomment to rename old id field
      if (data.id) {
        data.old_id = data.id;
        delete data.id;
      }

      for (let key in data) {
        // Convert timestamps
        if (fieldTypes.timestamp.includes(key)) {
          data[key] =
            data[key] && new Date(data[key]) != "Invalid Date"
              ? new Date(data[key])
              : null;
        }
      }

      // Add rowNum
      data.rowNum = String(rowCount + 1);

      // Rename created_at to createdAt
      if (data.created_at) {
        data.createdAt = data.created_at;
        delete data.created_at;
      }

      batches[commitCounter].set(ref, data);

      counter++;
    } else {
      counter = 0;
      commitCounter++;
      batches[commitCounter] = firestore.batch();
    }
  })
  .on("end", async () => {
    console.log(
      `[INFO] importing ${rowCount} rows to ${collectionName} collection..`
    );

    await commitBatches(batches);
    console.log("[INFO] import completed");
  });
ย 
listCollections.js
โ„น๏ธ Usage:
List name and size (number of documents) of all existing collections.
๐Ÿญ Sample output:
notion imagenotion image
๐Ÿ–‹ The script:
// Ignore item #0 and #1 of argv (node bin path and script path)
const [, , arg] = process.argv;

if (arg == "-h" || arg == "--help") {
  console.log("List name and size of all existing collections.");
  console.log("Usage\t: node listCollections.js");
  console.log("Example\t: node listCollections.js");
  process.exit(0);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

admin
  .firestore()
  .listCollections()
  .then((snapshot) => {
    console.log("fetching data..");

    let rows = [];
    snapshot.forEach((snaps) => {
      rows.push({
        collection: snaps["_queryOptions"].collectionId,
      });
    });

    if (!rows.length) {
      console.log(
        `[INFO] firestore database for project ${serviceAccount.project_id} did not have any collections yet`
      );
      return;
    }

    console.log(`${rows.length} collections found..`);

    (async () => {
      await Promise.all(
        rows.map(async ({ collection }, i) => {
          const size = await admin
            .firestore()
            .collection(collection)
            .get()
            .then((collectionSnapshot) => collectionSnapshot.size)
            .catch((err) =>
              console.log("[ERROR] could not get collection size:", err.message)
            );
          rows[i].size = size;
        })
      );
      console.table(rows);
    })();
  })
  .catch((error) => console.log(error));
listDocuments.js
โ„น๏ธ Usage:
List documents from given collection.
๐Ÿญ Sample output:
Error message when no fields are specified.Error message when no fields are specified.
Error message when no fields are specified.
Output are displayed using console.table for ease of reading.Output are displayed using console.table for ease of reading.
Output are displayed using console.table for ease of reading.
๐Ÿ–‹ The script:
// Ignore item #0 and #1 of argv (node bin path and script path)
const [, , collectionName, fields, limit = 100] = process.argv;

if (!collectionName || collectionName == "-h" || collectionName == "--help") {
  console.log(
    "List documents in given collection. Limit is set to 100 records by default."
  );
  console.log(
    "Usage\t: node listDocuments.js <collectionName> <fields> <limit=100>"
  );
  console.log("Example\t: node listDocuments.js users id,name,email");
  process.exit(0);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

admin
  .firestore()
  .collection(collectionName)
  .limit(limit)
  .get()
  .then((snapshot) => {
    let rows = [];

    if (snapshot.empty) {
      console.log("[INFO] collection did not contain any documents yet");
      return;
    }

    if (!fields) {
      console.log(
        "[ERROR] please specify fields to retrieve. possible fields:"
      );
      console.log(Object.keys(snapshot.docs[0].data()));
      return;
    }

    snapshot.forEach((doc) => {
      let data = {};
      fields.split(",").forEach((field) => {
        data.docId = doc.id;
        data[field] = doc.data()[field];
      });
      rows.push(data);
    });
    console.table(rows);
  });
listDocumentsTemplate.js
โ„น๏ธ Usage:
Unlike other scripts, this script is intended to be manually adjusted according to your use. You can use it when doing adhoc tasks that are not covered by other script. Add your code after the // Add your code here comment and then run the script. The script will iterate through the documents in collection you specify (similar to listDocuments.js) and run your custom code. When you want to use it for another different scenario, you can adjust your custom code and rerun it again.
๐Ÿญ Sample output:
(No sample output for this script)
๐Ÿ–‹ The script:
// Ignore item #0 and #1 of argv (node bin path and script path)
const [, , collectionName, fields, limit = 100] = process.argv;

if (!collectionName || collectionName == "-h" || collectionName == "--help") {
  console.log(
    "A script template that list documents in given collection and run custom code that you specify. Limit is set to 100 records by default."
  );
  console.log(
    "Usage\t: node listDocumentsTemplate.js <collectionName> <fields> <limit=100>"
  );
  console.log("Example\t: node listDocumentsTemplate.js users id,name,email");
  process.exit(0);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

admin
  .firestore()
  .collection(collectionName)
  .limit(limit)
  .get()
  .then((snapshot) => {
    let rows = [];

    if (snapshot.empty) {
      console.log("[INFO] collection did not contain any documents yet");
      return;
    }

    if (!fields) {
      console.log(
        "[ERROR] please specify fields to retrieve. possible fields:"
      );
      console.log(Object.keys(snapshot.docs[0].data()));
      return;
    }

    snapshot.forEach((doc) => {
      let data = {};
      fields.split(",").forEach((field) => {
        data.docId = doc.id;
        data[field] = doc.data()[field];
      });
      rows.push(data);
    });

    // Uncomment to display data
    // console.table(rows);

    // Add your code here
  });
putJson.js
โ„น๏ธ Usage:
Create new document based on given JSON
๐Ÿญ Sample output:
(No sample output at this moment)
๐Ÿ–‹ The script:
// Ignore item #0 and #1 of argv (node bin path and script path)
const [, , collectionName, json, id] = process.argv;

if (!collectionName || collectionName == "-h" || collectionName == "--help") {
  console.log("Create new document based on given JSON.");
  console.log("Usage\t: node putJson.js <collectionName> <json> <id=autoid>");
  console.log('Example\t: node putJson.js users \'{"name":"John Doe"}\'');
  process.exit(0);
}

if (!json) {
  console.log("[ERROR] json must be specified.");
  process.exit(1);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

const collectionRef = admin.firestore().collection(collectionName);

(id ? collectionRef.doc(id) : collectionRef.doc())
  .create(JSON.parse(json))
  .then((doc) => {
    console.log("[INFO] document created");
  })
  .catch((err) => {
    console.log("[ERROR] could not create document:", err.message);
  });
setClaims.js
โ„น๏ธ Usage:
Update custom claims of given Firebase user.
๐Ÿญ Sample output:
notion imagenotion image
๐Ÿ–‹ The script:
// Ignore first, second items of argv (node bin path and script path)
const [, , email, customClaims] = process.argv;

if (!email || email == "-h" || email == "--help") {
  console.log("Update custom claims of given Firebase user.");
  console.log("Usage\t: node setClaims.js <email> <customClaims>");
  console.log(
    "Example\t: node setClaims.js john@example.com '{\"isAdmin\":true}'"
  );
  process.exit(0);
}

if (!customClaims) {
  console.log("[ERROR] please specify one or more claims. example:");
  console.log(`node setClaims.js ${email} ADMIN,TECH`);
  process.exit(1);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

const auth = admin.auth();

auth
  .getUserByEmail(email)
  .then(async (user) => {
    await auth.setCustomUserClaims(user.uid, {
      ...user.customClaims,
      ...JSON.parse(customClaims),
    });

    // Reload user
    user = await auth.getUserByEmail(email);

    console.log("[INFO] custom claims updated:");
    console.log(user.toJSON().customClaims);
    process.exit();
  })
  .catch((err) => {
    console.log("[ERROR] could not get user:", err.message);
    process.exit(1);
  });
setRoles.js
โ„น๏ธ Usage:
Update roles value of Firebase user custom claims.
๐Ÿญ Sample output:
notion imagenotion image
๐Ÿ–‹ The script:
// Ignore first, second items of argv (node bin path and script path)
const [, , email, claims] = process.argv;

if (!email || email == "-h" || email == "--help") {
  console.log("Update `roles` value of Firebase user custom claims.");
  console.log("Usage\t: node setRoles.js <email> <claims>");
  console.log("Example\t: node setRoles.js john@example.com ADMIN,TECH");
  process.exit(0);
}

if (!claims) {
  console.log("[ERROR] please specify one or more claims. example:");
  console.log(`node setRoles.js ${email} ADMIN,TECH`);
  process.exit(1);
}

const admin = require("firebase-admin");
const serviceAccount = require(process.env.SERVICE_ACCOUNT_FILE);

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  databaseURL:
    process.env.FIRESTORE_DATABASE_URL ||
    `https://${serviceAccount.project_id}.firebaseio.com`,
});

const auth = admin.auth();

auth
  .getUserByEmail(email)
  .then(async (user) => {
    await auth.setCustomUserClaims(user.uid, {
      ...user.customClaims,
      roles: claims.split(","),
    });

    // Reload user
    user = await auth.getUserByEmail(email);

    console.log("[INFO] roles updated. customClaims is now:");
    console.log(user.toJSON().customClaims);
    process.exit();
  })
  .catch((err) => {
    console.log("[ERROR] could not get user:", err.message);
    process.exit(1);
  });
ย 
That's it, I hope it helps. Till next!
ย 

P.S:
Was this helpful?
ยท
โ€ฆ views

Loading Comments...