The Structured Package Format (SPF) is a simplified way to design and write Wolfram Language packages, particularly ones that comprise multiple source files. It does away with the need to make every source file its own separate package and manually manage contexts with functions like BeginPackage, EndPackage, Begin and End. Instead, the SPF lets you think in terms of three "scopes" for package symbols: Public (the symbols that you want to export from the package), Package (symbols that you want to be visible within all files of your package, but not visible to users) and Private (symbols that are private to the files in which they appear).

The SPF does not replace or change the existing package system. Think of it as a small framework that sits on top of the package system, handling the details for you.

Wolfram Language Packages

A package can be defined as a collection of code that exposes some set of Wolfram Language functionality via a context. When the user loads the package with Needs or Get, they can use the exported functions from the package in their session. This works because the package's context gets placed on the user's $ContextPath, which makes the exported (or "public") symbols from the package visible in the user's session. "Visible" means that the symbols can be used via just their short names (like SomeFunction), not requiring the fully decorated name (like SomePackage`SomeFunction).

The Basics of a Package

To understand how the Structured Package Format (SPF) works and why you might want to use it, it helps to review some things about Wolfram Language packages. The basics of packages are discussed in the documentation, particularly here and here, but what follows is a very quick review.

Here is a trivial package that exports only one function, MyPublicFunction.

BeginPackage["MyPackage`"]

(* Symbols that you want to export from the package are named here.
   They get created in the MyPackage` context, which will be on the user's
   $ContextPath after the package is loaded, so they can be used via their short names.
*)
MyPublicFunction

Begin["`Private`"]

(* Code and definitions go here.
   All symbols introduced in this section are private to the package, as they will
   be created in the MyPackage`Private` context, which is not on the user's $ContextPath.
*)

MyPublicFunction[x_] := privateFunction[x + 1]

privateFunction[y_] := y^2

End[]

EndPackage[]

A trivial example package.

Some of the things that happen as this package is loaded, line by line:

• The BeginPackage statement sets the current context to be MyPackage` and makes $ContextPath become just {"MyPackage`", "System`"}.

• The symbol PublicFunction is introduced, and because it is not in the System` context (the only other context on $ContextPath), it gets created in MyPackage`.

• Begin["Private`"] makes MyPackage`Private` the current context, and thus all symbols introduced after this point will get created in that private context.

• End[] ends the MyPackage`Private` context, restoring the current context to MyPackage`.

• EndPackage[] ends the MyPackage` context, restoring the current context to whatever it was prior to the BeginPackage. It also adds the MyPackage` context to $ContextPath, which is what allows users of the package to use exported symbols like PublicFunction without writing MyPackage`PublicFunction.

This is a simple system, but you have to write a lot of "boilerplate" code in the form of BeginPackage/EndPackage/Begin/End calls and generally manage contexts yourself. And things get much trickier when your package becomes complex enough that you want to split its implementation across multiple files. This is where the SPF steps in.

Packages with Multiple Files

Many small packages can be implemented as a single file, but for code organization and encapsulation purposes, you often want to split your code up across more than one file. If you were writing a package called InstrumentXYZ` for controlling a certain scientific instrument, you might decide to partition the code into multiple logical groupings, like InputOutput.wl, Analysis.wl, Visualization.wl, Utilities.wl, etc. Each of these source files might contribute something to the package's public functions and also provide functions that need to be called in other files of the package. For example, the Utilities.wl file will have a number of small internal utility functions that are used throughout the package.

A typical way to handle a package file structure like this is to make every file into its own "sub-package" of the top-level package context. This means that the InputOutput.wl file would begin with BeginPackage["InstrumentXYZ`InputOutput`"], and similarly for the other files. Each file gets its own full complement of BeginPackage/EndPackage/Begin/End statements and all the context-switching semantics that come with that. If any one of the files wants to use functions defined in any of the other files of the package, it must use Needs on the package context corresponding to the other file.

You also need a file that the main InstrumentXYZ` context maps to, which you can name init.wl (there are good reasons for picking that name, as discussed later). It would start with BeginPackage["InstrumentXYZ`"], then declare all the public symbols (meaning all the ones in the InstrumentXYZ` context), then call Needs on all the other contexts to trigger those files to load, and then end with EndPackage[].

This example represents a perfectly legitimate way to structure a package, but you wanted to create the single InstrumentXYZ` package and you ended up creating that one plus as many other packages as you have source files, and you had to spend a lot of time thinking about contexts, filenames, a web of Needs statements, and a difficult-to-discern file loading order.

The Structured Package Format

The SPF was created to provide a simple way to structure packages so that they are easy to design, write and maintain. It frees developers from having to manage a complex web of dependencies among the files of a package and eliminates a lot of the "boilerplate" code you typically have to type to create packages. It also makes it much easier to refactor packages, such as by adding files or moving code between files.

"Public", "Package" and "Private" Symbols

The core idea of the SPF is to recognize that for many packages, there are three convenient "scopes" for the symbols of the package, which can be called "Public", "Package" and "Private".

The Public scope is symbols that are "exported" from the package, meaning that the symbols are visible to users after the package has been loaded. This generally means that the symbols are in the package's main context, so they become visible in the user's session after the package is loaded with Needs or Get. In addition to being visible to users, Public symbols are visible within all the files of the package.

The Package scope is symbols that should be visible everywhere within all the files of the package, but not visible to users that load the package. This is where you put, say, internal utility functions that need to be used throughout the files of the package.

All symbols in a file that are not explicitly put into the Public or Package scopes are automatically in the Private scope. Private symbols can only be seen in the current file, so there is no possibility that an x in one file might conflict with an x in another file.

To put symbols into the Public or Package scopes, you use the functions PackageExported and PackageScoped, which you can think of as special types of "declarations".

Taking the MyPackage` example from above and rewriting it in SPF style produces this much shorter file:

PackageExported[MyPublicFunction]

MyPublicFunction[x_] := privateFunction[x + 1]

privateFunction[y_] := y^2

The MyPackage` example package from earlier written in SPF style.

The PackageExported[MyPublicFunction] line tells the SPF framework that the symbol MyPublicFunction should be "exported" from the package, which means created in the package's main context. All the other symbols in the file (x, y, and privateFunction) will be private to the file.

Note that there is no BeginPackage statement anywhere (nor EndPackage, Begin, End). You do not have to do context manipulation yourself in your source files.

There is a little bit more code than that to write, however, as the package still needs a loader file:

PackageInitialize["MyPackage`"]

The init.wl loader file for the MyPackage` example written in SPF style.

It is the call to PackageInitialize that handles all the scope management as package files are read in.

The PackageInitialize Function

The PackageInitialize function is the core of the implementation of the SPF. It sets up the package's context, reads in all the implementation files and performs all the context-management for you, including the BeginPackage/EndPackage calls. Every SPF package will have a file that calls PackageInitialize. This is called the "loader file" because it is the file that is responsible for reading in all the other files of your package.

Loader Files

Consider what happens when a user evaluates Get["SomePackage`"]. The system resolves the context SomePackage` to a file (more details about how this resolution is done are provided later), and then that file is read in. If that one file is the entire contents of the package, then the job is finished. But if a package is split up across several implementation files, then the file that is being read in by Get needs to ensure that all the other package files get read in as well.

Most multi-file packages have this concept of a "loader file", a file that is different from the others and primarily acts to ensure the other files get read in. In practice, these loader files can become quite complex, and a major goal of the SPF is to simplify loader files by encapsulating as much useful loader file functionality as possible within the PackageInitialize function.

In an SPF package, the loader file is the file that calls PackageInitialize, and PackageInitialize can be understood as the "loader function" to which you delegate the responsibility of ensuring your files all get loaded, the proper contexts are set up, etc.

What PackageInitialize Does

The details of how to control the behavior of PackageInitialize and how it works internally are discussed later, but think of it primarily as the function that names the package context and reads in the source files. A loader file that contains just this:

PackageInitialize["MyPackage`"]

when read in, says, in effect, "create the MyPackage` package and read in all the source files around me to supply the code for that package". The definition of "around me" is "alongside the loader file, and up to three levels deep beneath it in a directory hierarchy".

Note that for many packages, the call to PackageInitialize is the only place where the context of the package appears in the code.

Remember that PackageInitialize is not a function called by the users of your SPF package. They load your package with Needs like any other package. The call to PackageInitialize is just an implementation detail of your package. Users will not need to know or care whether you used the SPF or any other package development style.

PackageInitialize Needs a Directory

The loader file is, by definition, the file that has the call to PackageInitialize. Because PackageInitialize finds and loads all .wl (and .m) files "around" the loader file, this file needs to reside in a directory specific to your package, which of course would be standard practice anyway. Your loader file, say init.wl, will reside in a directory alongside the one or more other source files of your package. The concept of a "loose" loader file, disassociated from its intended directory of source files, is not meaningful, and if inadvertently executed could result in a large number of unintended .wl files being loaded, especially because PackageInitialize searches up to three levels deep in subdirectories for source files.

The "Declarations" PackageExported and PackageScoped

Earlier the functions PackageExported and PackageScoped were introduced as the special "declarations" that put symbols into the Public and Package scopes.

Some uses of PackageExported and PackageScoped.

One reason for calling them declarations, other than it being quite descriptive of what they do, is that they do not behave quite like normal functions. For example, they do nothing at evaluation time. Instead, they are collected from each file and analyzed during a special "scanning" step that happens within PackageInitialize. Because this scanning step does not evaluate anything in the file, the calls to PackageExported and PackageScoped must appear in the file in a way that they can be easily recognized. In practice, this means each one on a line by itself. In particular, do not try to "program" with them, as the following examples show:

(**** Allowed: ****)

PackageExported[Func1]

PackageExported[
   {Func1, Func2}
]

(**** NOT ALLOWED: ****)

1 + PackageExported[Func1]

PackageScoped /@ {func1, func2}

myPublicSymbols = {Function1, Function2, Function3}
PackageExported[myPublicSymbols] (* exports the one symbol myPublicSymbols *)

Some legal and illegal uses of PackageExported and PackageScoped.

Another consequence of the fact that the PackageExported and PackageScoped declarations are detected during a special scanning step is that it does not matter where in your files these declarations occur. You can put them near the definitions of the functions, collected near the top of each file, collected into a single file from all the other files, or any other arrangement. The order in which they appear does not matter, and you can have duplicates (calls to, say, PackageExported[SomeFunc] in multiple files, or more than once in the same file).

The PackageImport Declaration

If you need to use functions from other packages, you typically put Needs statements in your package to ensure that those other packages are loaded and made visible within your package. You are free to use Needs within an SPF package, and it works like it always has. But the SPF introduces a new PackageImport function for loading other packages that provides some extra functionality.

Some uses of PackageImport.

PackageImport has two basic usages: one where you import all the public symbols from the specified context, and one where you import only a subset of symbols ("import-by-name"). When used with just a context name, PackageImport imports all the symbols and is effectively just Needs. Also note that PackageImport, like Needs, will not reload packages if they have already been loaded in the session.

PackageImport is referred to above as a "declaration" because, like PackageExported and PackageScoped, it is examined during the special "scanning" pass that is part of the SPF. That means that PackageImport calls must appear as top-level expressions in the file, not as part of a "program" that evaluates to a PackageImport statement.

(**** Allowed: ****)

PackageImport["SomeOtherPackage`"]

PackageImport["SomeOtherPackage`",
   {Func1, Func2}
]

(**** NOT ALLOWED: ****)

PackageImport /@ {"Context1`", "Context2`"}

f[x_] := (
    PackageImport["SomeOtherContext`", SomeSymbol];
    SomeSymbol[x]
)

Some legal and illegal uses of PackageImport.

Unlike with PackageExported and PackageScoped, it does matter where in your files you put PackageImport calls. This is because PackageImport (like Needs) does have evaluation-time behavior—it loads an external package, changes the current $ContextPath, etc. If a file wants to use symbols from another package, there must be a call to PackageImport or Needs in that file before the first use of a symbol from the other package.

In an SPF file, the effect of PackageImport is limited to the file in which the call appears, and the same thing is true of calls to Needs. You cannot put a PackageImport statement in an early-loading file of your package and expect that later-loaded files will also be able to see symbols from that package. For every file that uses an external package, that file needs its own PackageImport declaration for that package. If you have an external package that is needed to be used in every file of your package, you can avoid putting PackageImport statements into every file by using the "HiddenImports" property of PackageInitialize.

Import by Name

The main feature that PackageImport provides compared to Needs is the ability to import only a subset of symbols from another package. There are several reasons why you might want to do that. For example, you might need to avoid a clash between a symbol in the other context and one of your own with the same name, or defend against conflicts if symbols get added to the other package in the future, or because you want your code to clearly declare all its dependencies down to the level of individual symbols.

Importing a subset of symbols from an external package does not mean that only the definitions of those functions get loaded. The entire external package gets loaded, as it must, but PackageImport ensures that only the named symbols from the other package are visible in the source file from that point onward.

The reference page for PackageImport has a simple example of importing a single function from an external package.

You can use multiple PackageImport calls within one file for different symbols from the same external package. An advanced detail of PackageImport is that when you have multiple PackageImport calls for the same context in the same file, they all get "coalesced" into one single call at the location of the first-appearing PackageImport for that context. For example, you might have three PackageImport statements scattered throughout a file, each naming a different symbol that you want to import from "Foo`". In this case, the point where "Foo`" is actually loaded is at the first PackageImport on "Foo`" in the file, as if you had included all the imported symbols in one single PackageImport call at that spot.

If you do not want to use import-by-name, then you might ask why you should use PackageImport instead of Needs. The answer is that, although it is true that PackageImport is essentially just Needs when not using import-by-name, declaring your external dependencies can be useful to tools that might exist in the future for analyzing packages.

Single-File Packages

Although the SPF is intended mainly as a way to simplify the development of multi-file packages, you can also use it to create a "single-file package" where the package's code follows the call to PackageInitialize in the same file. To create a single-file package, you use None as the second argument to PackageInitialize. The second argument is where you normally indicate a directory or file path, where it means "look here for the files that make up this package". But using None means "do not look at any other files; the code for this package resides entirely in the current file".

Here is a simple example of a single-file package:

PackageInitialize["Adders`", None]

(* In a single-file package (that is, where None is the second argument
   to PackageInitialize) you can put SPF-style code directly in the loader file.
*)
PackageExported[AddOne]

AddOne[x_] := x + 1

A trivial single-file package in SPF style.

Normally in an SPF package, the loader file, the file that calls PackageInitialize, is not allowed to have SPF-specific code like PackageExported calls. Those calls occur in the source files of the package, not the loader file itself. But in a single-file package, the rules are different, and you can use the full set of SPF features directly in the loader file.

A good way to think about how single-file packages work is to imagine that the file is sliced in two, right after the line that calls PackageInitialize. Everything after that cut behaves as if it was put into a separate source file and then loaded in the usual way as part of the PackageInitialize.

Creating a First SPF Package

To understand the features of SPF, it is helpful to work through the creation of a basic package. In this section a simple SPF style package named Adders` is created that exports some functions for adding numbers. The input cells in this section create all the directories and files on the fly. Of course, in actual practice, you would likely create and manage packages using your favorite editor, IDE, or the Package Editor built into the notebook front end.

A small extra step will also be taken to make a paclet out of the Adders` package. You can think of a paclet as just a wrapper around a package (or other types of content) that tells the system how to find the content.

The first thing to do is create a directory that will hold all the contents of the package. It is useful to make this directory have the same name as the package:

If you were only planning to make a package, you could put all the source files directly into the Adders directory. But you are making a paclet wrapper as well, and it is conventional for paclets to have their different types of content in specially named subdirectories. For Wolfram Language code, the standard directory name is Kernel, so create that directory:

It is into this Kernel subdirectory that all the package's source files will go.

The Adders` package exports only two functions, AddOne and AddTwo. For code organization purposes, the code for those functions is put into two separate files. It does not matter what you name these files, but here AddOne.wl and AddTwo.wl are chosen.

Now create the AddOne.wl file. You want AddOne to be exported from the package, so include a PackageExported[AddOne] statement:

Now create the AddTwo.wl file. You want AddTwo to be exported from the package, so include a PackageExported[AddTwo] statement:

Note that the definition of AddTwo contains calls to AddOne. This works because AddOne is declared as PackageExported in the AddOne.wl file, so it is visible in all files of the package, without any concern whether AddOne.wl loads before or after AddTwo.wl.

Now create the Logging.wl file, which contains the log function. That function is used in all files of the package but should not be exported from the package, hence declare it as PackageScoped:

Put the PackageExported and PackageScoped statements in these files alongside the definitions for the respective functions, but those statements could be moved anywhere, even collected into a separate file if desired.

To finish off the package, you must create the init.wl loader file, which calls PackageInitialize:

The Adders` package is now complete. You have an Adders directory that contains a Kernel subdirectory with all the source files including the init.wl file. Take a look at the directory and file layout:

If you copied the Adders directory to a location on $Path, like $UserBaseDirectory/Applications, then it could be loaded with Needs["Adders`"]. This is just a temporary package, so instead of copying it elsewhere, add its parent directory to $Path:

Now it can be loaded with Needs in the usual way:

You might be wondering how Needs["Adders`"] gets resolved to the init.wl loader file. More details about that topic are discussed in a later section, but for the case of a lookup based on $Path like this, the system looks for a directory that matches the context Adder` on $Path, and if it finds it, it looks inside to see if there is an init.wl file or if there is a Kernel subdirectory with an init.wl file, and if so, it loads that file. This special treatment in the system for files named init.wl (or init.m) goes back to the beginning of the Wolfram System and is one reason why it makes sense to name your loader file init.wl.

To see the file that a context resolves to, use the FindFile function:

FindFile is quite useful during development to ensure you are loading your package from the place you think you are.

Making the Paclet

A paclet is just a small wrapper around a bit of Wolfram functionality, with a package being the most common type of content. All you need to do to make something a paclet is to add an appropriate PacletInfo.wl file.

The PacletInfo.wl file goes into the package's top-level directory, alongside the Kernel directory. Now you see why the Kernel directory was created to hold the package's files—the PacletInfo.wl file is uncluttered in the top directory, and the package content is separated into the Kernel subdirectory.

The details of the paclet system are discussed elsewhere, but here is the content of a PacletInfo.wl file that has the minimum information needed to expose the Adders` package:

The name chosen for the paclet, Adders, does not need to match the context of the package, but that is a common choice. What matters is that the PacletInfo.wl file names the Adders` context, so the paclet system knows to look within that paclet for the package's loader file.

Now that you have created a paclet wrapper around your package, it is the paclet system that will allow the Adders` package to be found, not a lookup based on $Path. To prove that the package is being found via a paclet-based lookup, you must undo the addition of packageDir to $Path that you made earlier:

This paclet was not created in a location where the system automatically looks for paclets, so you have to tell the paclet system about this new location. The function that does that is PacletDirectoryLoad. You can think of PacletDirectoryLoad as the paclet world's counterpart to appending to $Path:

Note that PacletDirectoryLoad does not cause any files in the package to load, it just tells the system to inspect this directory for paclets it might contain.

The Adders` context will now be properly mapped to the package's init.wl file via the paclet system:

It should be emphasized that the paclet system's only involvement here is to find the Adders` package (that is, the path to its loader file). That process works in exactly the same way regardless of how the package is implemented.

A final note on terminology. It is easy to mix up the terms paclet and package, especially because many paclets consist of just a single package. A package is the name for the body of Wolfram Language code and the context it exposes. A paclet is what wraps the package and allows it to be shared, published, updated, etc. You do not really need to know anything about paclets to develop packages, but many packages will be distributed as paclets, so it often makes sense to develop your package as a paclet from the beginning.

SPF in Detail

The SPF does not change anything within the core package system. It still uses BeginPackage, EndPackage, Begin and End internally, you just do not have to write those calls yourself. That is why you can consider it a framework that sits on top of the package system. If you think that the SPF does not meet your needs, you do not have to use it for your own packages. It does not break any existing paradigms for package development, so you can still design packages in whatever style you like.

Two-Pass Reading of Files

SPF is a file-based system, meaning that its functions are meant to be embedded in Wolfram Language source files, not called from an interactive session. A package will be represented by a directory hierarchy of one or more source files. The system maps the MyPackage` context to your loader file, and then the PackageInitialize call in your loader file collects up all the source files to read and operates on them.

This tutorial has already mentioned a special "scanning" step that is performed on your source files prior to any of them actually being loaded. This two-pass reading technique is at the core of how the SPF works.

The First Pass: Scanning for Declarations

PackageInitialize gathers up all the files that it thinks need to be loaded, in the order it thinks they should be loaded. Before loading any of them, it initiates a scanning pass that reads each file, looking for the so-called "declarations" PackageExported, PackageScoped and PackageImport. This pass reads one expression at a time from the file, without evaluating it, and if that unevaluated expression has head PackageExported, PackageScoped or PackageImport, then it is recognized and recorded. It is because these declarations must be recognized without evaluating anything that each one must appear verbatim as a top-level expression in the file.

At the end of the first pass, every file has been scanned and the SPF knows enough about what your package exports and imports that it can perform all the necessary context manipulation as the files are actually loaded in the second pass.

The Second Pass: Loading and Evaluating

In the second pass, the files are read and evaluated via calls to Get, the standard function for reading Wolfram Language files. Before each file is read with Get, there is some context and $ContextPath manipulation to make all the appropriate contexts and symbols visible, and after the call to Get, those manipulations are undone. Because each file is read with a reset $ContextPath, changes to $ContextPath within a file, like from a call to PackageImport or Needs, do not persist beyond the file in which they appear. Therefore you cannot load an external package in one file and expect that package's symbols to be visible in files of your own package that are loaded later in sequence.

Because the second pass is a standard evaluation pass on the files, the order in which they are read can matter. PackageInitialize has several properties that you can use to control what files get loaded and in what order. One of the advantages of SPF is that the loading order of the files is easy to control, whereas with an old style "web of Needs" type of package structure, the actual loading order might not be obvious to discern and subject to change by innocuous-looking updates in Needs calls among your files.

What Are the Actual Contexts?

One of the main conveniences of SPF is to free you from having to think about the specific contexts of your symbols and instead think just in terms of Public/Package/Private "scopes". But sometimes you need to know the actual contexts of your symbols. For example, you might want to write tests that invoke your Package-scope functions directly, or you might be debugging your package in a notebook session and want to be able to call its Private functions.

If your package context is MyPackage`:

• The PackageExported symbols will be in the MyPackage` context.

• The PackageScoped symbols will be in the MyPackage`PackageScope` context.

• All other symbols will be in a context that is unique to that file. This context is built by concatenating the package context, a segment based on the file name and Private`. If the package has a file named Utilities.wl, then the private context for that file will be MyPackage`Utilities`Private`. If the Utilities.wl file is nested in an Analysis subdirectory, then the context will be MyPackage`Analysis`Utilities`Private`.

Some Details of the Loader File

Many packages can have one-line loader files, but sometimes you want some code to be evaluated before or after the call to PackageInitialize. It is possible to put code into a loader file before or after the PackageInitialize call, but you have to be aware of what the current $Context will be:

• Before the PackageInitialize call, the current context is whatever it was when the call to load your package occurred, which is likely to be Global`.

• After the PackageInitialize call, the current context is whatever it was before the PackageInitialize call, and also YourPackage` has been added to $ContextPath.

Note that these rules for the current context are exactly the same as for an old-style package file in the regions before the BeginPackage call and after the matching EndPackage[].

Here is an example of a loader file that is probably not doing what the programmer wants:

loaderFileDir = DirectoryName[$InputFileName]
If[StringStartsQ[loaderFileDir, $HomeDirectory], Print["file is in my home dir"]]

PackageInitialize["MyPackage`"]

A flawed loader file.

The problem with the above file is that when it executes it creates a loaderFileDir symbol in whatever is the current $Context (Global`, perhaps). You do not want your package to clutter the user's session with an "escaped" symbol from your loader file. One way to fix this would be to everywhere refer to loaderFileDir via a fully qualified context name, like MyPackage`Private`loaderFileDir. If you have more than a few symbols you need to create, then the technique of using a full context name for every symbol becomes awkward. A better method for that case is to put in a BeginPackage/EndPackage pair yourself:

BeginPackage["MyPackage`"]
Begin["`Private`"]

(***
 Any code you put here will automatically have its symbols created in the MyPackage`Private` context.
***)
loaderFileDir = DirectoryName[$InputFileName]
If[StringStartsQ[loaderFileDir, $HomeDirectory], Print["file is in my home dir"]]

End[]
EndPackage[]

PackageInitialize["MyPackage`"]

An improved loader file.

You can use the same technique to insert code after the call to PackageInitialize.

Before you start cluttering your loader file with code, however, consider whether that is what you really need to do. Many package authors have a simpler need—to control the order in which their package files get loaded. For example, if you have some code that needs to be loaded before any other files in your package, you can use the "LoadFirstFiles" property of PackageInitialize to make that file load first, or the "LoadLastFiles" property to ensure a file gets loaded after all your other files:

PackageInitialize["MyPackage`", <|"LoadFirstFiles" -> "LoadMeFirst.wl", "LoadLastFiles" -> "LoadMeLast.wl"|>]

It is important to remember that you cannot put SPF functionality into the loader file itself. (There is a special exception to this rule, the feature of single-file packages, but that case is ignored here.) In other words, you cannot put, say, calls to PackageExported in the loader file. A way to think about this is that SPF is really just "what happens during the execution of PackageInitialize". Before the call to PackageInitialize and after it returns, you are not in the "SPF world", so calls like PackageExported and PackageScoped do not have any meaning and will trigger warning messages.

An advantage of using "LoadFirstFiles" to ensure that a specific file gets loaded ahead of the others, compared to trying to put initialization code into the loader file itself, is that your file of initialization code is loaded by PackageInitialize and therefore can use SPF functionality within it. All the magic of SPF happens only among the files that get loaded by PackageInitialize, not in the file that is itself calling PackageInitialize.

Your loader file does not have to be called init.wl. You might call it MyPackage.wl or MyPackageLoader.wl or anything else. Remember, though, that it is up to you to ensure that the system will map your context name to your loader file. Calling the loader file init.wl engages some automatic behavior that makes the system more likely to find the correct file, as discussed here. It also makes the identity of the loader file instantly recognizable to you or anyone who examines your package.

Advanced PackageInitialize Usage

Controlling Files to be Read

A common need for package developers is to control the order in which the files of the package get loaded. For example, you might have code in one file that needs to evaluate before a certain other file gets loaded, or you might have some code you want executed after all the other package files have been read in.

Because PackageInitialize takes over the reading of your package's source files, you need to be able to tell it what files get read and in what order. You control how PackageInitialize works primarily through the use of a set of properties that are given as an Association in the last argument to PackageInitialize. The properties that control file reading are:

The "LoadFirstFiles" and "LoadLastFiles" properties are likely to be the most useful:

PackageInitialize["MyPackage`", <|"LoadFirstFiles" -> "LoadMeFirst.wl", "LoadLastFiles" -> "LoadMeLast.wl"|>]

The "IgnoreFiles" property lets you specify a file or set of files that should be completely ignored, as if they did not exist. If, during development, you wanted to temporarily exclude a certain file from the package, you could change your PackageInitialize call to:

PackageInitialize["MyPackage`", <|"IgnoreFiles" -> "Experimental.wl"|>]

The "SkipLoadingFiles" property lets you specify a file or set of files that should not be loaded but should be scanned so that their PackageScoped and PackageExported declarations are still recognized and therefore contribute those symbols to the package. This is an advanced property that is likely to be used only by developers who are familiar with the internals of SPF.

PackageInitialize["MyPackage`", <|"SkipLoadingFiles" -> "Experimental.wl"|>]

Automatically Protecting Symbols

Developers often want the public symbols from their package Protected after it loads. The "SymbolsToProtect" property lets you specify what symbols you want to give this treatment to, which includes:

• Unprotecting the symbols before your package loads (in case the package has already been loaded and left the symbols Protected)

• Making the symbols Protected and ReadProtected after the package loads

Note that Clear is not called on the symbols at the start of loading, as that might be considered too invasive for general use.

Here are some values for the "SymbolsToProtect" property:

	None	no symbols protected (default)
	All	all public symbols protected
	{"sym1", "sym2",...}	protect the named symbols
	Automatic	all public symbols and paclet autoload symbols are protected (advanced users)

This is what your PackageInitialize call might look like if you wanted all your public symbols Protected:

PackageInitialize["MyPackage`", <|"SymbolsToProtect" -> All|>]

Using the Second Argument to PackageInitialize

PackageInitialize takes an optional second argument that you can use to tell it where to find the files it should load. In typical usage, PackageInitialize is called from your package's loader file, and PackageInitialize searches the directory in which that loader file resides to find the files to load for the package. You do not need to specify where the files for the package are to be found because the location is taken to be "around the loader file". But there are times during development and debugging when it can be useful to control where the package files should be found.

If you worked through the Creating a First SPF Package section earlier, you can run this command to reload the Adders` package:

When you call PackageInitialize by pointing it at a specific directory, it automatically skips loading any init.wl files in that directory (if you are keeping count, that is one more reason to name your loader file init.wl). You are in effect bypassing the loader file in the package's directory and executing your own ad hoc loader.

A reason you might want to do that is if you are experimenting with properties in your PackageInitialize call and do not want to change your init.wl file every time. Here, load the Adders` package again but change the file order:

Single-File Packages

Another way to use the second argument to PackageInitialize is to specify None, which engages a special mode that allows all the package's code to reside within the loader file itself—a so-called "single-file package". Using None as the second argument means "do not look at any other files; all the code for the package is to be found in this loader file itself". Single-file packages are discussed in more detail here.

Single-File Reload

Sometimes during development you might want to reload only a single file in your package, to test some changes you just made in that file. The package has already been loaded in your session in the normal way, and now you want to reload a single modified file. You cannot just call Get on that modified file directly, as SPF files need to be loaded by PackageInitialize to function properly. For a single-file reload, use the second argument of PackageInitialize to point to the file you want reloaded.

If you worked through the Creating a First SPF Package section earlier, you can run this command to reload just the AddTwo.wl file from the Adders` package:

As a general guideline, package authors should strive to make their packages load quickly, not doing any time-consuming operations at load time. For packages that are quick to load, single-file reload is not very useful—just call Get["MyPackage`"] and reload the whole package every time you edit anything. But in some cases reloading an entire package might be time-consuming, or maybe you specifically do not want anything in the other package files to reload.

SPF in Notebooks

The SPF is fundamentally a feature for Wolfram Language source files (.wl and .m). What would it even mean to execute PackageExported[Foo] at an input prompt? In fact, it gives a warning:

One consequence of this is that you cannot use an interactive notebook as the source code for your package. For example, you cannot have a notebook with an input cell that calls PackageExported[Foo] followed by an input cell with the definition for the function Foo, then select both cells and do a Shift+Enter evaluation. Recall that PackageInitialize makes two separate reading passes over files, and this makes it simply incompatible with Shift+Enter evaluations.

What you can do from a notebook with SPF is to call PackageInitialize. Normally, PackageInitialize is the function that is called in your loader file to load and process the files of your package, but you can call it from an input prompt in an interactive session. There are a few cases during development and debugging where you might want to call PackageInitialize directly.

When you call PackageInitialize from a notebook cell, there is no obvious location from which to collect the files for the package (you probably do not mean to read in all the .wl files that happen to be alongside the current notebook file and up to three levels deep!), so you need to use the second argument to PackageInitialize to point at the location of the package files. The Using the Second Argument to PackageInitialize section details cases where you might want to call PackageInitialize directly, outside of a loader file.

How Packages Are Found: Paclets and $Path

Once you have created your package, you want users to be able to load it in the usual way: a call to Needs on the package's context. It is up to you as a package developer to ensure that Needs or Get will correctly map your context to your loader file. This context-to-filename mapping happens in two phases. First, the paclet system is given a chance to find an appropriate file for the context. If that fails, a $Path based lookup is used.

The earlier section Creating a First SPF Package shows how to make a simple SPF package and also how to set things up so that it can be found by a call to Needs on the context. It shows how to make the package available via a $Path based lookup and also when wrapped in a paclet.

Remember that the rules for how your package gets found have nothing directly to do with the style in which the package is implemented. That means this entire section is not about SPF specifically, just about general guidelines for how packages are found.

The rules are easier to state and understand in the case of $Path based lookups, so that will be the starting point. Some extra rules will also be ignored that pertain to .mx files (which are discussed here).

When looking for SomeContext` based on $Path, the system will, for each directory on $Path in order, look for:

• a SomeContext.wl file

• a SomeContext directory with an init.wl file in it

• a SomeContext directory with a Kernel subdirectory that has an init.wl file in it

If the context has multiple segments in it, like Foo`Bar`, the system looks on $Path for a Foo directory with a Bar.wl file in it.

These rules tell you that there is special logic for init.wl (and init.m) files, another reason it makes sense to name your loader file init.wl.

Paclet-based lookups adhere to the same general logic about init.wl files, but the overall behavior is a little more complex because the paclet system is more flexible and lets you map a context to any file you like within your paclet. A Kernel extension in your PacletInfo.wl file is how you announce to the paclet system that your paclet provides a specific context. A full treatment of how Kernel extensions work in the paclet system is beyond the scope of this tech note, but here is a simple example of a PacletInfo.wl file for a hypothetical paclet called MyPaclet that provides a single package called MyPackage`:

PacletObject[<|
    "Name" -> "MyPaclet",
    "Version" -> "1.0",
    "Extensions" -> {
        {"Kernel", "Root" -> "Kernel", "Context" -> "MyPackage`"}
    }
|>]

A PacletInfo.wl file for a paclet that provides the MyPackage` package.

The Kernel extension names the context MyPackage`, so the paclet system knows to look within this paclet for the package. The "Root"->"Kernel" property says that the file(s) for this package reside in a Kernel subdirectory of the paclet's top directory (the directory that contains the PacletInfo.wl file). What file from the Kernel directory will the paclet system choose to load? If there is an init.wl file, that will be chosen. If not, the system looks for a MyPackage.wl file (the filename matching the context). If that file is not present, the lookup will fail. You would need a slightly more elaborate PacletInfo.wl file to map the MyPackage` context to a file not named init.wl or MyPackage.wl.

If your paclet is installed into the standard paclet location with PacletInstall, Needs["YourPackage`"] will work. But while you are developing your paclet, you do not have it installed, so you need a way to tell the paclet system where to look for it. The function that tells the paclet system about new places to look for local paclets is PacletDirectoryLoad. A typical developer workflow is to use PacletDirectoryLoad on your paclet's top-level development directory. Then you can edit the package files, repeatedly loading the package into the session with Get["MyPackage`"]. Remember that PacletDirectoryLoad is not "sticky" across sessions, so you must call it again in each new session.

The FindFile function is very useful during development to determine what file the system thinks a specific context maps to. You can see where a package would load from without actually loading it: