Developer Reference
First off, please make sure you have read and understood our code of conduct. We expect everybody to adhere to it. You can find this project’s code of condact here.
To get started with developing, fork the github repository and clone it into a local directory. If this is your first time contributing to an open-source project, have a look at these general guidelines.
This project uses fairly tight restrictions in terms of testing and linting. Don’t be discouraged if you run into issues. Always feel free to ask questions by raising an issue. Many style guides and ideas here are taken from the Hypermodern Python blog created by Claudio Jolowicz. These blog post, while intense, are an excellent read and are highly recommended.
Dependencies
For full testing of the project,
you should have the supported python versions installed.
Furthermore, you need to install nox
,
which can be done from the console by typing:
$ pip install nox
If you completely test your setup with nox
,
dependency installation is not required.
If you like to test directly with pytest
,
write your own temporary routines,
create examples, etc.,
you can install the dependencies
from your console by typing:
$ pip install -r requirements.txt
$ pip install -r dev-requirements.txt
Contribution requirements
All code submissions should be tested. The CI requires that all unit tests and lint tests complete successfully before merging into the main branch is allowed.
Coverage:
All code should be tested. Code testing coverage of 100% is required to ensure future integrity.
Docstrings:
The documentation automatically generates the API reference from the supplied docstrings. Please use Sphinx style docstrings to document your routines.
Linting:
All code must adhere to flake8
specifications,
see also Linting.
This allows for better readability.
Even though you won’t remember all linting rules,
you should go back and fix linting issues after testing.
The tests will give you feedback on what to do.
Test driven development
Testing of the iniabu
package is done using pytest
and automated using nox
.
To run a full test using nox
,
Python 3.6, 3.7, 3.8, and 3.9,
must be available in the environment.
A full nox test, which includes
linting, safety, and tests
can be run from the terminal by typing:
$ nox
To check wha sessions are implemented, run the following code from your terminal:
$ nox -l
This will also display information
for all sessions implemented in nox
.
You can also check out the noxfile.py
directly.
Please also check the nox documentation for further options, etc.
The test suite lives in the tests
folder.
This folder mirrors the package structure.
In addition,
a file named conftest.py
is used to set fixtures for pytest.
This allows for proper initialization
of the package with every test.
The iniabu project requires that the whole code base is covered with tests, i.e., that a code coverage of 100% is maintained. Of course, the tests should also be meaningful! This code coverage ensures that future developments do not break other functionalities. More about this can be found in Testing.
Example: Bugfix
If a bug is found in the code and reported, a bug fix should be implemented in the following way:
Write a test with the wanted outcome and make sure the test suite fails due to the reported bug.
Fix the bug in the source code.
The bug is fixed once tew new test passes successfully and no other tests were broken.
Formatting with black
The iniabu project adopts the default style that is provided by the black python formatter. Their GitHub site describes in detail how to use the formatter. There is really not much to configure.
Formatting with black is implemented via a pre-commit hooks, see section Pre-commit hooks for more information. The hook file also specifies the used version of black.
Linting
Linting heavily improves code readability.
Please follow all linting guidelines.
We use flake8
.
Furthermore, the following additional plugins are used:
flake8-bandit
to identify security issues.flake8-black
to check that the codebase is formatted using black.flake8-bugbear
to find additional bugs and design problems.flake8-docstrings
to ensure docsting completeness and consistency.flake8-import-order
to ensure consistent package importing.
Exact linting options are configured in the
.flake8
file.
This file also contains comments
to better understand the options.
Invoking only linting with nox can be done from the terminal by typing:
$ nox -rs lint
To fix linting issues,
read the output of the linter carefully.
If absolutely required,
use the # noqa: err
comment
after the line in question
to exclude specific linting errors.
Replace the err
part with the error number
that was returned by the linter.
This should only be used where it makes sense.
Testing
Project testing is done with pytest
.
The following pytest
plugins
are defined in the dev-requirements.txt
file:
pytest-cov
to test code coverage.pytest-mock
to mock out certain parts of the code base.pytest-sugar
to display nicely formatted output.
The pytest.ini
file configures
the testing environment properly.
To run tests from the terminal,
assuming that all dependencies are installed,
type:
$ pytest
To test the test suite only with nox
,
you can type the following into the terminal:
$ nox -rs tests
Again, adding the option -p 3.9
would limit the test to
Python 3.9 only.
Hypothesis
Where adequate, make use of the hypothesis package for writing your tests. Have a look at the existing tests for input on what to test for. Hypothesis allows for simple edge case testing and often catches errors that might otherwise go through.
Docstring example testing
As discussed before, docstrings should be used to document every new routine. The docstrings should also contain examples. Check out the source code for examples on how to write them.
Examples should of course represent the behavior of the code. It thus must be written in Python prompt form. For example, look at the following example:
>>> from iniabu import ini # loads with default ("lodders09")
>>> ini.database = "nist" # change database to "nist"
>>> ini.database
'nist'
To ensure that all examples are correct,
they can be tested using
xdoctest.
This is implemented as a nox
session
and can be called
by typing the following into your terminal:
$ nox -rs xdoctest
Note: This is not part of the unit tests
and must be called separately.
A GitHub action is implemented
to specifically run doctests
.
Safety
Safety
is used to check all required dependencies
for known security vulnerabilities.
To run only safety
form nox
,
type the following into your terminal:
$ nox -rs safety
Documentation
The documentation uses sphinx
.
It is automatically built and hosted by
readthedocs.io.
To locally build the documentation,
run the following from your terminal:
$ nox -rs docs
This will dump the html
files
for the documentation into the
docs/_build
folder.
You can now locally browse them.
Pre-commit hooks
Using pre-commit hooks your project can be tested for simple formatting mishaps. These will also be automatically corrected. Here, we use the pre-commit framework. If you want to set up pre-commit hooks, go to the folder and run the following command (after installing pre-commit using pip or pipx):
$ pre-commit install
This will install the hooks that are defined in .pre-commit-config.yaml into your git repository. Note that a fairly standard pre-commit configuration is used. Black is pinned to a specific version, i.e., the same version as in the nox file itself.
Structure of the data tables
All data lives in the data
subfolder
underneath the main package.
Aside from the nist.py
file,
all databases contain 2 dictionaries,
one for elements and one for isotopes.
Missing values must be denotes as np.nan
.
ele_dict
Element dictionary
The element dictionary ele_dict
is shaped in the following structure:
ele_dict = {
'Symb':
[
sol_abu_ele,
[a1, ..., an],
[rel_abu1, ..., rel_abun],
[sol_abu1, ..., sol_abun]
],
...
}
Here, Symb
is the element symbol,
e.g., H
for hydrogen.
This is the dictionary key.
The entry is followed by a list.
The entry sol_abu_ele
is the
solar abundance of the element in number fractions
normalized such that the solar abundance of Si is 1e6.
a1
to an
are the atomic mass numbers
of the isotopes of this element.
rel_abu1
to rel_abun
and sol_abu1
to sol_abun
are these isotopes relative abundances and solar abundance,
respectively.
Note that the relative abundances
must be normed such that their sum is unity.
iso_dict
Isotope dictionary
The isotope dictionary iso_dict
is shaped in the following structure:
iso_dict = {
'Symb-A':
[
rel_abu,
sol_abu
],
...
}
Here, Symb-A is the key of the dictionary
and is composed of the element symbol Symb
and the isotope’s atomic number A
.
A dash separates the two entries.
The dictionary entries are rel_abu
and sol_abu
,
which are the isotopes relative and
solar abundance, respectively.
The same normalization rules apply as discussed above.
Adding a database
Parser files for individual databases
that have already been added
were put into the dev
folder in the repository.
Every database added has their datafile in some format
and a parser living there.
The parser creates automatically the python file.
Have a look at some of these parsers,
especially the write method.
Here, the headers,
imports, etc. are written.
Then the dictionaries are dumped out using json.dump()
.
While this results in a really ugly format for the python file,
running black
over the generated file
will properly format everything.
This python file must then be moved to the iniabu/data
folder.
Adjust the iniabu/data/__init__.py
file
to contain imports for the two new dictionaries.
Extend the database_selector()
function
with an additional elif
statement
to contain the new database.
Finally, new tests for this database must be added.
All tests live in the test
folder,
which has the same structure as the iniabu
folder
that contains the package source code.
One good way to write a test is to use an existing test
file for a dataset.
Then adjust the subroutines and associated asserts.
At least make sure that tests exist for:
Data integrity
Solar abundance of Si is 106
Relative abundances of all isotopes sum to unity
Finally, add a new test in test_main.py
to ensure that the database loads correctly.
You should add a consistency check for the new database.
This ensures that code
coverage stays at 100%.