This commit is contained in:
2026-07-14 22:50:46 +04:00
parent 27143319e3
commit bd19e0682b
3116 changed files with 467189 additions and 0 deletions

View File

@@ -0,0 +1,4 @@
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test,examples,integration_test}/**/*.{ex,exs}"]
]

Binary file not shown.

View File

@@ -0,0 +1,112 @@
# Changelog
## v2.9.0 (2026-01-10)
* Enhancements
* Whenever possible, have error messages that include a PID also include ancestors and one of process label, name, or initial call
* Allow clients to retry when specified
* Remove sensitive options before calling `after_connect`
## v2.8.1 (2025-06-24)
* Enhancements
* Automatically set and read process labels in error reports
## v2.8.0 (2025-06-24)
* Enhancements
* Allow `unallow_existing` as an opt to `ownership_allow/4`
* Improve ETS performance by enabling descentralized counters
* Increase default queue interval to 2000ms
## v2.7.0 (2024-07-02)
* Enhancements
* Add API for retrieving pool metrics
* Include a built-in listener that emits telemetry events
* Bug fixes
* Discard EXIT messages from trapped exits
## v2.6.0 (2023-10-15)
* Enhancements
* Call `disconnect` on terminate
* Allow `handle_begin` callbacks to return query for logging purposes
* Add `:connection_listeners_tag`
* Add `DBConnection.available_connection_options/0`
* Add `DBConnection.available_start_options/0`
## v2.5.0 (2023-04-10)
* Internal changes
* No longer depend on `connection`
## v2.4.3 (2022-11-22)
* Bug fixes
* Fix bug where `disconnect_all/2` interval would be disabled above 4294ms
* Add `:idle_limit` to limit the amount of disconnections on a ping
## v2.4.2 (2022-03-03)
* Enhancements
* Add `DBConnection.connection_module/1`
## v2.4.1 (2021-10-14)
* Enhancements
* Add `DBConnection.disconnect_all/2`
## v2.4.0 (2021-04-02)
* Enhancements
* Add telemetry events for connection errors
* Use `:rand` default algorithm
* Allow decentralized lookups on DBConnection.Ownership
## v2.3.1 (2020-11-25)
* Enhancements
* Add `:connection_listeners` to `DBConnection.start_link/2`
* Allow connection `~> 1.0`
## v2.3.0 (2020-10-14)
This release requires Elixir v1.7+.
* Bug fixes
* Fix deprecation warnings related to the use of `System.stacktrace()`
## v2.2.2 (2020-04-22)
* Bug fixes
* Make sure all idle connections in the pool are pinged on each idle interval
## v2.2.1 (2020-02-04)
* Enhancements
* Remove warnings
## v2.2.0 (2019-12-11)
* Enhancements
* Add `:idle_time` to `DBConnection.LogEntry`
* Ping all stale connections on idle interval
* Add `crash_reason` to relevant Logger error reports
* Ping all stale connections on idle interval. One possible downside of this approach is that we may shut down all connections at once and if there is a request around this time, the response time will be higher. However, this is likely better than the current approach, where we ping only the first one, which means we can have a pool of stale connections. The current behaviour is the same as in v1.0
## v2.1.1 (2019-07-17)
* Enhancements
* Reduce severity in client exits to info
* Improve error message on redirect checkout
* Bug fixes
* Make sure ownership timeout is respected on automatic checkouts
## v2.1.0 (2019-06-07)
* Enhancements
* Require Elixir v1.6+
* Include client stacktrace on check out timeouts

View File

@@ -0,0 +1,100 @@
# DBConnection
Database connection behaviour and database connection pool designed for
handling transaction, prepare/execute, cursors and client process
describe/encode/decode.
Examples of using the `DBConnection` behaviour are available in
`./examples/db_agent/` and `./examples/tcp_connection/`.
There is also [a series of articles on building database adapters](http://blog.plataformatec.com.br/2018/11/building-a-new-mysql-adapter-for-ecto-part-i-hello-world/). It includes articles covering both DBConnection and Ecto integrations.
## Contributing
Run unit tests with:
$ mix test
To run the integration tests (for each available pool):
$ mix test.pools
To run all tests:
$ mix test.all
## Design
This library is made of four main modules:
* `DBConnection` - this is the code running on the client
and the specification of the DBConnection API
* `DBConnection.Connection` - this is the process that
establishes the database connection
* `DBConnection.ConnectionPool` - this is the connection
pool. A client asks the connection pool for a connection.
There is also an ownership pool, used mostly during tests,
which we won't discuss here.
* `DBConnection.Holder` - the holder is responsible for
keeping the connection and checkout state. It is modelled
by using an ETS table.
Once a connection is created, it creates a holder and
assigns the connection pool as the heir. Then the holder
is promptly given away to the pool. The connection itself
is mostly a dummy. It is there to handle connections and pings.
The state itself (such as the socket) is all in the holder.
Once there is a checkout, the pool gives the holder to the
client process and stores all relevant information in the
holder table itself. If the client terminates without
checking in, then the holder is given back to the pool via
the heir mechanism. The pool will then discard the connection.
One important design detail in DBConnection is that it avoids
copying data. Other database libraries would send a request
to the connection process, perform the query in the connection
process, and then send it back to the client. This means a lot of
data copying in Elixir. DBConnection keeps the socket in the
holder and works on it directly.
DBConnection also takes all of the care necessary to handle
failures, and it shuts down the connection and the socket
whenever the client does not check in the connection to avoid
recycling sockets/connections in a corrupted state (such as a socket
that is stuck inside a transaction).
### Deadlines
When a checkout happens, a deadline is started by the client
to send a message to the pool after a time interval. If the
deadline is reached and the connection is still checked out,
the holder is deleted and the connection is terminated. If the
client tries to use a terminated connection, an error will
be raised (see `Holder.handle/4`).
### Pool
The queuing algorithm used by the pool is [CoDel](https://queue.acm.org/appendices/codel.html)
which allows us to plan for overloads and reject requests
without clogging the pool once checkouts do not read a certain
target.
## License
Copyright 2015 James Fish
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,33 @@
{<<"links">>,
[{<<"GitHub">>,<<"https://github.com/elixir-ecto/db_connection">>}]}.
{<<"name">>,<<"db_connection">>}.
{<<"version">>,<<"2.9.0">>}.
{<<"description">>,
<<"Database connection behaviour for database transactions and connection pooling">>}.
{<<"elixir">>,<<"~> 1.11">>}.
{<<"files">>,
[<<"lib">>,<<"lib/db_connection">>,<<"lib/db_connection/util.ex">>,
<<"lib/db_connection/backoff.ex">>,<<"lib/db_connection/log_entry.ex">>,
<<"lib/db_connection/ownership.ex">>,
<<"lib/db_connection/connection_pool">>,
<<"lib/db_connection/connection_pool/pool.ex">>,
<<"lib/db_connection/task.ex">>,<<"lib/db_connection/connection_error.ex">>,
<<"lib/db_connection/query.ex">>,<<"lib/db_connection/holder.ex">>,
<<"lib/db_connection/ownership">>,
<<"lib/db_connection/ownership/proxy.ex">>,
<<"lib/db_connection/ownership/manager.ex">>,
<<"lib/db_connection/watcher.ex">>,<<"lib/db_connection/pool.ex">>,
<<"lib/db_connection/connection_pool.ex">>,
<<"lib/db_connection/connection.ex">>,
<<"lib/db_connection/telemetry_listener.ex">>,
<<"lib/db_connection/app.ex">>,<<"lib/db_connection.ex">>,
<<".formatter.exs">>,<<"mix.exs">>,<<"README.md">>,<<"CHANGELOG.md">>]}.
{<<"app">>,<<"db_connection">>}.
{<<"licenses">>,[<<"Apache-2.0">>]}.
{<<"requirements">>,
[[{<<"name">>,<<"telemetry">>},
{<<"app">>,<<"telemetry">>},
{<<"optional">>,false},
{<<"requirement">>,<<"~> 0.4 or ~> 1.0">>},
{<<"repository">>,<<"hexpm">>}]]}.
{<<"build_tools">>,[<<"mix">>]}.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,23 @@
defmodule DBConnection.App do
@moduledoc false
use Application
@impl true
def start(_type, _args) do
children = [
{Task.Supervisor, name: DBConnection.Task},
dynamic_supervisor(DBConnection.Ownership.Supervisor),
dynamic_supervisor(DBConnection.ConnectionPool.Supervisor),
DBConnection.Watcher
]
Supervisor.start_link(children, strategy: :one_for_all, name: __MODULE__)
end
defp dynamic_supervisor(name) do
Supervisor.child_spec(
{DynamicSupervisor, name: name, strategy: :one_for_one},
id: name
)
end
end

View File

@@ -0,0 +1,111 @@
defmodule DBConnection.Backoff do
# This module provides a functional abstraction over backoffs with different types. It exposes
# a struct and a couple of functions to work with it.
@moduledoc false
@compile :nowarn_deprecated_function
alias DBConnection.Backoff
@default_type :rand_exp
@min 1_000
@max 30_000
@type t :: %__MODULE__{
type: :stop | :rand | :exp | :rand_exp,
min: non_neg_integer(),
max: non_neg_integer(),
state: term()
}
defstruct [:type, :min, :max, :state]
@spec new(keyword) :: t | nil
def new(opts) when is_list(opts) do
case Keyword.get(opts, :backoff_type, @default_type) do
:stop ->
nil
type ->
{min, max} = min_max(opts)
new(type, min, max)
end
end
@spec backoff(t) :: {non_neg_integer, t}
def backoff(backoff)
def backoff(%Backoff{type: :rand, min: min, max: max} = s) do
{rand(min, max), s}
end
def backoff(%Backoff{type: :exp, min: min, state: nil} = s) do
{min, %{s | state: min}}
end
def backoff(%Backoff{type: :exp, max: max, state: prev} = s) do
next = min(Bitwise.<<<(prev, 1), max)
{next, %{s | state: next}}
end
def backoff(%Backoff{type: :rand_exp, max: max, state: state} = s) do
{prev, lower} = state
next_min = min(prev, lower)
next_max = min(prev * 3, max)
next = rand(next_min, next_max)
{next, %{s | state: {next, lower}}}
end
@spec reset(t) :: t
def reset(backoff)
def reset(%Backoff{type: :rand} = s), do: s
def reset(%Backoff{type: :exp} = s), do: %{s | state: nil}
def reset(%Backoff{type: :rand_exp, min: min, state: {_, lower}} = s) do
%{s | state: {min, lower}}
end
## Internal
defp min_max(opts) do
case {opts[:backoff_min], opts[:backoff_max]} do
{nil, nil} -> {@min, @max}
{nil, max} -> {min(@min, max), max}
{min, nil} -> {min, max(min, @max)}
{min, max} -> {min, max}
end
end
defp new(_, min, _) when not (is_integer(min) and min >= 0) do
raise ArgumentError, "minimum #{inspect(min)} not 0 or a positive integer"
end
defp new(_, _, max) when not (is_integer(max) and max >= 0) do
raise ArgumentError, "maximum #{inspect(max)} not 0 or a positive integer"
end
defp new(_, min, max) when min > max do
raise ArgumentError, "minimum #{min} is greater than maximum #{max}"
end
defp new(:rand, min, max) do
%Backoff{type: :rand, min: min, max: max, state: nil}
end
defp new(:exp, min, max) do
%Backoff{type: :exp, min: min, max: max, state: nil}
end
defp new(:rand_exp, min, max) do
lower = max(min, div(max, 3))
%Backoff{type: :rand_exp, min: min, max: max, state: {min, lower}}
end
defp new(type, _, _) do
raise ArgumentError, "unknown type #{inspect(type)}"
end
defp rand(min, max) do
:rand.uniform(max - min + 1) + min - 1
end
end

View File

@@ -0,0 +1,536 @@
defmodule DBConnection.Connection do
@moduledoc false
@behaviour :gen_statem
require Logger
alias DBConnection.Backoff
alias DBConnection.Holder
alias DBConnection.Util
@timeout 15_000
@sensitive_opts [:parameters, :hostname, :port, :username, :password, :database]
@doc false
def start_link(mod, opts, pool, tag) do
start_opts = Keyword.take(opts, [:debug, :spawn_opt])
:gen_statem.start_link(__MODULE__, {mod, opts, pool, tag}, start_opts)
end
@doc false
def child_spec(mod, opts, pool, tag, child_opts) do
Supervisor.child_spec(
%{id: __MODULE__, start: {__MODULE__, :start_link, [mod, opts, pool, tag]}},
child_opts
)
end
@doc false
def disconnect({pid, ref}, err, state) do
:gen_statem.cast(pid, {:disconnect, ref, err, state})
end
@doc false
def stop({pid, ref}, err, state) do
:gen_statem.cast(pid, {:stop, ref, err, state})
end
@doc false
def ping({pid, ref}, state) do
:gen_statem.cast(pid, {:ping, ref, state})
end
## gen_statem API
@doc false
@impl :gen_statem
def callback_mode, do: :handle_event_function
@doc false
@impl :gen_statem
def init({mod, opts, pool, tag}) do
pool_index = Keyword.get(opts, :pool_index)
label = if pool_index, do: "db_conn_#{pool_index}", else: "db_conn"
Util.set_label(label)
s = %{
mod: mod,
opts: opts,
state: nil,
client: :closed,
pool: pool,
tag: tag,
timer: nil,
backoff: Backoff.new(opts),
connection_listeners: Keyword.get(opts, :connection_listeners, []),
after_connect: Keyword.get(opts, :after_connect),
after_connect_timeout: Keyword.get(opts, :after_connect_timeout, @timeout)
}
{:ok, :no_state, s, {:next_event, :internal, {:connect, :init}}}
end
@impl :gen_statem
def handle_event(type, info, state, s)
def handle_event(:internal, {:connect, _info}, :no_state, s) do
%{mod: mod, opts: opts, backoff: backoff, after_connect: after_connect} = s
try do
apply(mod, :connect, [connect_opts(opts)])
rescue
e ->
{e, stack} = maybe_sanitize_exception(e, __STACKTRACE__, opts)
reraise e, stack
else
{:ok, state} when after_connect != nil ->
ref = make_ref()
:gen_statem.cast(self(), {:after_connect, ref})
{:keep_state, %{s | state: state, client: {ref, :connect}}}
{:ok, state} ->
backoff = backoff && Backoff.reset(backoff)
ref = make_ref()
:gen_statem.cast(self(), {:connected, ref})
{:keep_state, %{s | state: state, client: {ref, :connect}, backoff: backoff}}
{:error, err} when is_nil(backoff) ->
Logger.error(
fn ->
[
inspect(mod),
" (",
Util.inspect_pid(self()),
") failed to connect: " | Exception.format_banner(:error, err, [])
]
end,
crash_reason: {err, []}
)
raise err
{:error, err} ->
Logger.error(
fn ->
[
inspect(mod),
?\s,
?(,
Util.inspect_pid(self()),
") failed to connect: "
| Exception.format_banner(:error, err, [])
]
end,
crash_reason: {err, []}
)
{timeout, backoff} = Backoff.backoff(backoff)
{:keep_state, %{s | backoff: backoff}, {{:timeout, :backoff}, timeout, nil}}
end
end
def handle_event(:internal, {:disconnect, {log, err}}, :no_state, %{mod: mod} = s) do
if log == :log do
severity =
case err do
%DBConnection.ConnectionError{severity: severity} -> severity
_ -> :error
end
Logger.log(severity, fn ->
[
inspect(mod),
?\s,
?(,
Util.inspect_pid(self()),
") disconnected: " | Exception.format_banner(:error, err, [])
]
end)
:ok
end
%{state: state, client: client, timer: timer, backoff: backoff} = s
demonitor(client)
cancel_timer(timer)
:ok = apply(mod, :disconnect, [err, state])
s = %{s | state: nil, client: :closed, timer: nil}
notify_connection_listeners(:disconnected, s)
case client do
_ when backoff == nil ->
{:stop, {:shutdown, err}, s}
{_, :after_connect} ->
{timeout, backoff} = Backoff.backoff(backoff)
{:keep_state, %{s | backoff: backoff}, {{:timeout, :backoff}, timeout, nil}}
_ ->
{:keep_state, s, {:next_event, :internal, {:connect, :disconnect}}}
end
end
def handle_event({:timeout, :backoff}, _content, :no_state, s) do
{:keep_state, s, {:next_event, :internal, {:connect, :backoff}}}
end
def handle_event(:cast, {:ping, ref, state}, :no_state, %{client: {ref, :pool}, mod: mod} = s) do
case apply(mod, :ping, [state]) do
{:ok, state} ->
pool_update(state, s)
{:disconnect, err, state} ->
{:keep_state, %{s | state: state}, {:next_event, :internal, {:disconnect, {:log, err}}}}
end
end
def handle_event(:cast, {:disconnect, ref, err, state}, :no_state, %{client: {ref, _}} = s) do
{:keep_state, %{s | state: state}, {:next_event, :internal, {:disconnect, {:log, err}}}}
end
def handle_event(:cast, {:stop, ref, err, state}, :no_state, %{client: {ref, _}} = s) do
{_, stack} = :erlang.process_info(self(), :current_stacktrace)
case err do
ok when ok in [:normal, :shutdown] ->
:ok
{:shutdown, _term} ->
:ok
_ ->
reason =
case err do
%{__exception__: true} -> Exception.format_banner(:error, err, stack)
_other -> "** #{inspect(err)}"
end
format =
~c"** State machine ~p terminating~n" ++
~c"** Reason for termination ==~n" ++
~c"~s~n"
:error_logger.format(format, [self(), reason])
end
{:stop, {err, stack}, %{s | state: state}}
end
def handle_event(:cast, {tag, _, _, _}, :no_state, s) when tag in [:disconnect, :stop] do
handle_timeout(s)
end
def handle_event(:cast, {:after_connect, ref}, :no_state, %{client: {ref, :connect}} = s) do
%{
mod: mod,
state: state,
after_connect: after_connect,
after_connect_timeout: timeout,
opts: opts
} = s
notify_connection_listeners(:connected, s)
case apply(mod, :checkout, [state]) do
{:ok, state} ->
opts = [timeout: timeout] ++ opts
opts = Keyword.drop(opts, @sensitive_opts)
{pid, ref} = DBConnection.Task.run_child(mod, state, after_connect, opts)
timer = start_timer(pid, timeout)
s = %{s | client: {ref, :after_connect}, timer: timer, state: state}
{:keep_state, s}
{:disconnect, err, state} ->
{:keep_state, %{s | state: state}, {:next_event, :internal, {:disconnect, {:log, err}}}}
end
end
def handle_event(:cast, {:after_connect, _}, :no_state, _s) do
:keep_state_and_data
end
def handle_event(:cast, {:connected, ref}, :no_state, %{client: {ref, :connect}} = s) do
%{mod: mod, state: state} = s
notify_connection_listeners(:connected, s)
case apply(mod, :checkout, [state]) do
{:ok, state} ->
pool_update(state, s)
{:disconnect, err, state} ->
{:keep_state, %{s | state: state}, {:next_event, :internal, {:disconnect, {:log, err}}}}
end
end
def handle_event(:cast, {:connected, _}, :no_state, _s) do
:keep_state_and_data
end
def handle_event(
:info,
{:DOWN, ref, _, pid, reason},
:no_state,
%{client: {ref, :after_connect}} = s
) do
message =
"client #{Util.inspect_pid(pid)} exited: " <> Exception.format_exit(reason)
err = DBConnection.ConnectionError.exception(message)
{:keep_state, %{s | client: {nil, :after_connect}},
{:next_event, :internal, {:disconnect, {down_log(reason), err}}}}
end
def handle_event(:info, {:DOWN, mon, _, pid, reason}, :no_state, %{client: {ref, mon}} = s) do
message =
"client #{Util.inspect_pid(pid)} exited: " <> Exception.format_exit(reason)
err = DBConnection.ConnectionError.exception(message)
{:keep_state, %{s | client: {ref, nil}},
{:next_event, :internal, {:disconnect, {down_log(reason), err}}}}
end
def handle_event(
:info,
{:timeout, timer, {__MODULE__, pid, timeout}},
:no_state,
%{timer: timer} = s
)
when is_reference(timer) do
message =
"client #{Util.inspect_pid(pid)} timed out because it checked out " <>
"the connection for longer than #{timeout}ms"
exc =
case Process.info(pid, :current_stacktrace) do
{:current_stacktrace, stacktrace} ->
message <>
"\n\n#{Util.inspect_pid(pid)} was at location:\n\n" <>
Exception.format_stacktrace(stacktrace)
_ ->
message
end
|> DBConnection.ConnectionError.exception()
{:keep_state, %{s | timer: nil}, {:next_event, :internal, {:disconnect, {:log, exc}}}}
end
def handle_event(
:info,
{:"ETS-TRANSFER", holder, _pid, {msg, ref, extra}},
:no_state,
%{client: {ref, :after_connect}, timer: timer} = s
) do
{_, state} = Holder.delete(holder)
cancel_timer(timer)
s = %{s | timer: nil}
case msg do
:checkin -> handle_checkin(state, s)
:disconnect -> handle_event(:cast, {:disconnect, ref, extra, state}, :no_state, s)
:stop -> handle_event(:cast, {:stop, ref, extra, state}, :no_state, s)
end
end
# We discard EXIT messages which may arrive if the process is trapping exits
def handle_event(:info, {:EXIT, _, _}, :no_state, s) do
handle_timeout(s)
end
def handle_event(:info, msg, :no_state, %{mod: mod} = s) do
Logger.info(fn ->
[inspect(mod), ?\s, ?(, Util.inspect_pid(self()), ") missed message: " | inspect(msg)]
end)
handle_timeout(s)
end
@doc false
@impl :gen_statem
# If client is :closed then the connection was previously disconnected
# and cleanup is not required.
def terminate(_, _, %{client: :closed}), do: :ok
def terminate(reason, _, s) do
%{mod: mod, state: state} = s
msg = "connection exited: " <> Exception.format_exit(reason)
msg
|> DBConnection.ConnectionError.exception()
|> mod.disconnect(state)
end
@doc false
@impl :gen_statem
def format_status(info, [_, :no_state, %{client: :closed, mod: mod}]) do
case info do
:normal -> [{:data, [{~c"Module", mod}]}]
:terminate -> mod
end
end
def format_status(info, [pdict, :no_state, %{mod: mod, state: state}]) do
case function_exported?(mod, :format_status, 2) do
true when info == :normal ->
normal_status(mod, pdict, state)
false when info == :normal ->
normal_status_default(mod, state)
true when info == :terminate ->
{mod, terminate_status(mod, pdict, state)}
false when info == :terminate ->
{mod, state}
end
end
## Helpers
defp maybe_sanitize_exception(e, stack, opts) do
if Keyword.get(opts, :show_sensitive_data_on_connection_error, false) do
{e, stack}
else
message =
"connect raised #{inspect(e.__struct__)} exception#{sanitized_message(e)}. " <>
"The exception details are hidden, as they may contain sensitive data such as " <>
"database credentials. You may set :show_sensitive_data_on_connection_error " <>
"to true when starting your connection if you wish to see all of the details"
{RuntimeError.exception(message), cleanup_stacktrace(stack)}
end
end
defp sanitized_message(%KeyError{} = e), do: ": #{Exception.message(%{e | term: nil})}"
defp sanitized_message(_), do: ""
defp connect_opts(opts) do
case Keyword.get(opts, :configure) do
{mod, fun, args} ->
apply(mod, fun, [opts | args])
fun when is_function(fun, 1) ->
fun.(opts)
nil ->
opts
end
end
defp down_log(:normal), do: :nolog
defp down_log(:shutdown), do: :nolog
defp down_log({:shutdown, _}), do: :nolog
defp down_log(_), do: :log
defp handle_timeout(s), do: {:keep_state, s}
defp demonitor({_, mon}) when is_reference(mon) do
Process.demonitor(mon, [:flush])
end
defp demonitor({mon, :after_connect}) when is_reference(mon) do
Process.demonitor(mon, [:flush])
end
defp demonitor({_, _}), do: true
defp demonitor(nil), do: true
defp start_timer(_, :infinity), do: nil
defp start_timer(pid, timeout) do
:erlang.start_timer(timeout, self(), {__MODULE__, pid, timeout})
end
defp cancel_timer(nil), do: :ok
defp cancel_timer(timer) do
case :erlang.cancel_timer(timer) do
false -> flush_timer(timer)
_ -> :ok
end
end
defp flush_timer(timer) do
receive do
{:timeout, ^timer, {__MODULE__, _, _}} ->
:ok
after
0 ->
raise ArgumentError, "timer #{inspect(timer)} does not exist"
end
end
defp handle_checkin(state, s) do
%{backoff: backoff, client: client} = s
backoff = backoff && Backoff.reset(backoff)
demonitor(client)
pool_update(state, %{s | client: nil, backoff: backoff})
end
defp pool_update(state, %{pool: pool, tag: tag, mod: mod} = s) do
case Holder.update(pool, tag, mod, state) do
{:ok, ref} ->
{:keep_state, %{s | client: {ref, :pool}, state: state}, :hibernate}
:error ->
{:stop, {:shutdown, :no_more_pool}, s}
end
end
defp normal_status(mod, pdict, state) do
try do
mod.format_status(:normal, [pdict, state])
catch
_, _ ->
normal_status_default(mod, state)
else
status ->
status
end
end
defp normal_status_default(mod, state) do
[{:data, [{~c"Module", mod}, {~c"State", state}]}]
end
defp terminate_status(mod, pdict, state) do
try do
mod.format_status(:terminate, [pdict, state])
catch
_, _ ->
state
else
status ->
status
end
end
defp cleanup_stacktrace(stack) do
case stack do
[{_, _, arity, _} | _rest] = stacktrace when is_integer(arity) ->
stacktrace
[{mod, fun, args, info} | rest] when is_list(args) ->
[{mod, fun, length(args), info} | rest]
end
end
defp notify_connection_listeners(action, %{} = state) do
%{connection_listeners: connection_listeners} = state
{listeners, message} =
case connection_listeners do
listeners when is_list(listeners) ->
{listeners, {action, self()}}
{listeners, tag} when is_list(listeners) ->
{listeners, {action, self(), tag}}
end
Enum.each(listeners, &send(&1, message))
end
end

View File

@@ -0,0 +1,24 @@
defmodule DBConnection.ConnectionError do
@moduledoc """
A generic connection error exception.
The raised exception might include the reason which would be useful
to programmatically determine what was causing the error.
"""
@typedoc since: "2.7.0"
@type t() :: %__MODULE__{
message: String.t(),
reason: :error | :queue_timeout,
severity: Logger.level()
}
defexception [:message, severity: :error, reason: :error]
@doc false
def exception(message, reason) when is_binary(message) and reason in [:error, :queue_timeout] do
message
|> exception()
|> Map.replace!(:reason, reason)
end
end

View File

@@ -0,0 +1,391 @@
defmodule DBConnection.ConnectionPool do
@moduledoc """
The default connection pool.
The queueing algorithm is based on [CoDel](https://queue.acm.org/appendices/codel.html).
You're not supposed to call any functions on this pool directly, but only pass this
as the value of the `:pool` option in functions such as `DBConnection.start_link/2`.
`disconnect_all/3`, which by default will result in connections being
reestablished, can be called periodically to recycle checked-in connections
after a maximum lifetime is reached. `Ecto SQL` users may find it at
https://hexdocs.pm/ecto_sql/Ecto.Adapters.SQL.html#disconnect_all/3
"""
use GenServer
alias DBConnection.Holder
alias DBConnection.Util
@behaviour DBConnection.Pool
@queue_target 50
@queue_interval 2000
@idle_interval 1000
@time_unit 1000
@doc false
def start_link({mod, opts}) do
GenServer.start_link(__MODULE__, {mod, opts}, start_opts(opts))
end
@doc false
@impl DBConnection.Pool
def checkout(pool, callers, opts) do
Holder.checkout(pool, callers, opts)
end
@doc false
@impl DBConnection.Pool
def disconnect_all(pool, interval, _opts) do
GenServer.call(pool, {:disconnect_all, interval}, :infinity)
end
@doc false
@impl DBConnection.Pool
def get_connection_metrics(pool) do
GenServer.call(pool, :get_connection_metrics, :infinity)
end
## GenServer api
@impl GenServer
def init({mod, opts}) do
DBConnection.register_as_pool(mod)
queue = :ets.new(__MODULE__.Queue, [:protected, :ordered_set, decentralized_counters: true])
ts = {System.monotonic_time(), 0}
{:ok, _} = DBConnection.ConnectionPool.Pool.start_supervised(queue, mod, opts)
target = Keyword.get(opts, :queue_target, @queue_target)
interval = Keyword.get(opts, :queue_interval, @queue_interval)
idle_interval = Keyword.get(opts, :idle_interval, @idle_interval)
idle_limit = Keyword.get_lazy(opts, :idle_limit, fn -> Keyword.get(opts, :pool_size, 1) end)
now_in_native = System.monotonic_time()
now_in_ms = System.convert_time_unit(now_in_native, :native, @time_unit)
codel = %{
target: target,
interval: interval,
delay: 0,
slow: false,
next: now_in_ms,
poll: nil,
idle_interval: idle_interval,
idle_limit: idle_limit,
idle: nil
}
codel = start_idle(now_in_native, start_poll(now_in_ms, now_in_ms, codel))
{:ok, {:busy, queue, codel, ts}}
end
@impl GenServer
def handle_call(:get_connection_metrics, _from, {status, queue, _, _} = state) do
{ready_conn_count, checkout_queue_length} =
case status do
:busy ->
{0, :ets.select_count(queue, [{{{:_, :_, :_}}, [], [true]}])}
:ready ->
{:ets.select_count(queue, [{{{:_, :_}}, [], [true]}]), 0}
end
metrics = %{
source: {:pool, self()},
ready_conn_count: ready_conn_count,
checkout_queue_length: checkout_queue_length
}
{:reply, [metrics], state}
end
def handle_call({:disconnect_all, interval}, _from, {type, queue, codel, _ts}) do
ts = {System.monotonic_time(), interval}
{:reply, :ok, {type, queue, codel, ts}}
end
@impl GenServer
def handle_info(
{:db_connection, from, {:checkout, _caller, now, queue?}},
{:busy, queue, _, _} = busy
) do
case queue? do
true ->
:ets.insert(queue, {{now, System.unique_integer(), from}})
{:noreply, busy}
false ->
message = "connection not available and queuing is disabled"
err = DBConnection.ConnectionError.exception(message)
Holder.reply_error(from, err)
{:noreply, busy}
end
end
def handle_info(
{:db_connection, from, {:checkout, _caller, _now, _queue?}} = checkout,
{:ready, queue, _codel, _ts} = ready
) do
case :ets.first(queue) do
{queued_in_native, holder} = key ->
Holder.handle_checkout(holder, from, queue, queued_in_native) and :ets.delete(queue, key)
{:noreply, ready}
:"$end_of_table" ->
handle_info(checkout, put_elem(ready, 0, :busy))
end
end
def handle_info({:"ETS-TRANSFER", holder, pid, queue}, {_, queue, _, _} = data) do
message = "client #{Util.inspect_pid(pid)} exited"
err = DBConnection.ConnectionError.exception(message: message, severity: :info)
Holder.handle_disconnect(holder, err)
{:noreply, data}
end
def handle_info({:"ETS-TRANSFER", holder, _, {msg, queue, extra}}, {_, queue, _, ts} = data) do
case msg do
:checkin ->
owner = self()
case :ets.info(holder, :owner) do
^owner ->
{time, interval} = ts
if Holder.maybe_disconnect(holder, time, interval) do
{:noreply, data}
else
handle_checkin(holder, extra, data)
end
:undefined ->
{:noreply, data}
end
:disconnect ->
Holder.handle_disconnect(holder, extra)
{:noreply, data}
:stop ->
Holder.handle_stop(holder, extra)
{:noreply, data}
end
end
def handle_info({:timeout, deadline, {queue, holder, pid, len}}, {_, queue, _, _} = data) do
# Check that timeout refers to current holder (and not previous)
if Holder.handle_deadline(holder, deadline) do
message =
"client #{Util.inspect_pid(pid)} timed out because " <>
"it queued and checked out the connection for longer than #{len}ms"
exc =
case Process.info(pid, :current_stacktrace) do
{:current_stacktrace, stacktrace} ->
message <>
"\n\n#{Util.inspect_pid(pid)} was at location:\n\n" <>
Exception.format_stacktrace(stacktrace)
_ ->
message
end
|> DBConnection.ConnectionError.exception()
Holder.handle_disconnect(holder, exc)
end
{:noreply, data}
end
def handle_info({:timeout, poll, {time, last_sent}}, {_, _, %{poll: poll}, _} = data) do
{status, queue, codel, ts} = data
# If no queue progress since last poll check queue
case :ets.first(queue) do
{sent, _, _} when sent <= last_sent and status == :busy ->
delay = time - sent
timeout(delay, time, queue, start_poll(time, sent, codel), ts)
{sent, _, _} ->
{:noreply, {status, queue, start_poll(time, sent, codel), ts}}
_ ->
{:noreply, {status, queue, start_poll(time, time, codel), ts}}
end
end
def handle_info({:timeout, idle, past_in_native}, {_, _, %{idle: idle}, _} = data) do
{status, queue, %{idle_limit: limit} = codel, ts} = data
drop_idle(past_in_native, limit, status, queue, codel, ts)
end
defp drop_idle(past_in_native, limit, status, queue, codel, ts) do
with true <- status == :ready and limit > 0,
{queued_in_native, holder} = key when queued_in_native <= past_in_native <-
:ets.first(queue) do
:ets.delete(queue, key)
Holder.maybe_disconnect(holder, elem(ts, 0), 0) or Holder.handle_ping(holder)
drop_idle(past_in_native, limit - 1, status, queue, codel, ts)
else
_ ->
{:noreply, {status, queue, start_idle(System.monotonic_time(), codel), ts}}
end
end
defp timeout(delay, time, queue, codel, ts) do
case codel do
%{delay: min_delay, next: next, target: target, interval: interval}
when time >= next and min_delay > target ->
codel = %{codel | slow: true, delay: delay, next: time + interval}
drop_slow(time, target * 2, queue)
{:noreply, {:busy, queue, codel, ts}}
%{next: next, interval: interval} when time >= next ->
codel = %{codel | slow: false, delay: delay, next: time + interval}
{:noreply, {:busy, queue, codel, ts}}
_ ->
{:noreply, {:busy, queue, codel, ts}}
end
end
defp drop_slow(time, timeout, queue) do
min_sent = time - timeout
match = {{:"$1", :_, :"$2"}}
guards = [{:<, :"$1", min_sent}]
select_slow = [{match, guards, [{{:"$1", :"$2"}}]}]
for {sent, from} <- :ets.select(queue, select_slow) do
drop(time - sent, from)
end
:ets.select_delete(queue, [{match, guards, [true]}])
end
defp handle_checkin(holder, now_in_native, {:ready, queue, _, _} = data) do
:ets.insert(queue, {{now_in_native, holder}})
{:noreply, data}
end
defp handle_checkin(holder, now_in_native, {:busy, queue, codel, ts}) do
now_in_ms = System.convert_time_unit(now_in_native, :native, @time_unit)
case dequeue(now_in_ms, holder, queue, codel, ts) do
{:busy, _, _, _} = busy ->
{:noreply, busy}
{:ready, _, _, _} = ready ->
:ets.insert(queue, {{now_in_native, holder}})
{:noreply, ready}
end
end
defp dequeue(time, holder, queue, codel, ts) do
case codel do
%{next: next, delay: delay, target: target} when time >= next ->
dequeue_first(time, delay > target, holder, queue, codel, ts)
%{slow: false} ->
dequeue_fast(time, holder, queue, codel, ts)
%{slow: true, target: target} ->
dequeue_slow(time, target * 2, holder, queue, codel, ts)
end
end
defp dequeue_first(time, slow?, holder, queue, codel, ts) do
%{interval: interval} = codel
next = time + interval
case :ets.first(queue) do
{sent, _, from} = key ->
:ets.delete(queue, key)
delay = time - sent
codel = %{codel | next: next, delay: delay, slow: slow?}
go(delay, from, time, holder, queue, codel, ts)
:"$end_of_table" ->
codel = %{codel | next: next, delay: 0, slow: slow?}
{:ready, queue, codel, ts}
end
end
defp dequeue_fast(time, holder, queue, codel, ts) do
case :ets.first(queue) do
{sent, _, from} = key ->
:ets.delete(queue, key)
go(time - sent, from, time, holder, queue, codel, ts)
:"$end_of_table" ->
{:ready, queue, %{codel | delay: 0}, ts}
end
end
defp dequeue_slow(time, timeout, holder, queue, codel, ts) do
case :ets.first(queue) do
{sent, _, from} = key when time - sent > timeout ->
:ets.delete(queue, key)
drop(time - sent, from)
dequeue_slow(time, timeout, holder, queue, codel, ts)
{sent, _, from} = key ->
:ets.delete(queue, key)
go(time - sent, from, time, holder, queue, codel, ts)
:"$end_of_table" ->
{:ready, queue, %{codel | delay: 0}, ts}
end
end
defp go(delay, from, time, holder, queue, %{delay: min} = codel, ts) do
case Holder.handle_checkout(holder, from, queue, 0) do
true when delay < min ->
{:busy, queue, %{codel | delay: delay}, ts}
true ->
{:busy, queue, codel, ts}
false ->
dequeue(time, holder, queue, codel, ts)
end
end
defp drop(delay, from) do
message = """
[#{ancestor()}] connection not available and request was dropped from queue after #{delay}ms. \
This means requests are coming in and your connection pool cannot serve them fast enough. \
You can address this by:
1. Ensuring your database is available and that you can connect to it
2. Tracking down slow queries and making sure they are running fast enough
3. Increasing the pool_size (although this increases resource consumption)
4. Allowing requests to wait longer by increasing :queue_target and :queue_interval
See DBConnection.start_link/2 for more information
"""
err = DBConnection.ConnectionError.exception(message, :queue_timeout)
Holder.reply_error(from, err)
end
defp ancestor do
Process.get(:"$ancestors", []) |> Enum.find(&is_atom/1)
end
defp start_opts(opts) do
Keyword.take(opts, [:name, :spawn_opt])
end
defp start_poll(now, last_sent, %{interval: interval} = codel) do
timeout = now + interval
poll = :erlang.start_timer(timeout, self(), {timeout, last_sent}, abs: true)
%{codel | poll: poll}
end
defp start_idle(now_in_native, %{idle_interval: interval} = codel) do
timeout = System.convert_time_unit(now_in_native, :native, :millisecond) + interval
idle = :erlang.start_timer(timeout, self(), now_in_native, abs: true)
%{codel | idle: idle}
end
end

View File

@@ -0,0 +1,33 @@
defmodule DBConnection.ConnectionPool.Pool do
@moduledoc false
use Supervisor, restart: :temporary
def start_supervised(tag, mod, opts) do
DBConnection.Watcher.watch(
DBConnection.ConnectionPool.Supervisor,
{DBConnection.ConnectionPool.Pool, {self(), tag, mod, opts}}
)
end
def start_link(arg) do
Supervisor.start_link(__MODULE__, arg)
end
@impl true
def init({owner, tag, mod, opts}) do
size = Keyword.get(opts, :pool_size, 1)
if size < 1, do: raise(ArgumentError, "pool size must be greater or equal to 1, got #{size}")
children = for id <- 1..size, do: conn(owner, tag, id, mod, opts)
sup_opts = [strategy: :one_for_one] ++ Keyword.take(opts, [:max_restarts, :max_seconds])
Supervisor.init(children, sup_opts)
end
## Helpers
defp conn(owner, tag, id, mod, opts) do
child_opts = [id: {mod, owner, id}] ++ Keyword.take(opts, [:shutdown])
DBConnection.Connection.child_spec(mod, [pool_index: id] ++ opts, owner, tag, child_opts)
end
end

View File

@@ -0,0 +1,448 @@
defmodule DBConnection.Holder do
@moduledoc false
require Record
alias DBConnection.Util
@queue true
@timeout 15000
@time_unit 1000
Record.defrecord(:conn, [:connection, :module, :state, :lock, :ts, deadline: nil, status: :ok])
Record.defrecord(:pool_ref, [:pool, :reference, :deadline, :holder, :lock])
@type t :: :ets.tid()
@type checkin_time :: non_neg_integer() | nil
## Holder API
@spec new(pid, reference, module, term) :: t
def new(pool, ref, mod, state) do
# Insert before setting heir so that pool can't receive empty table
holder = :ets.new(__MODULE__, [:public, :ordered_set, decentralized_counters: true])
conn = conn(connection: self(), module: mod, state: state, ts: System.monotonic_time())
true = :ets.insert_new(holder, conn)
:ets.setopts(holder, {:heir, pool, ref})
holder
end
@spec update(pid, reference, module, term) :: {:ok, t} | :error
def update(pool, ref, mod, state) do
holder = new(pool, ref, mod, state)
try do
:ets.give_away(holder, pool, {:checkin, ref, System.monotonic_time()})
{:ok, holder}
rescue
ArgumentError -> :error
end
end
@spec delete(t) :: {module, term}
def delete(holder) do
[conn(module: module, state: state)] = :ets.lookup(holder, :conn)
:ets.delete(holder)
{module, state}
end
## Pool API (invoked by caller)
@callback checkout(pool :: GenServer.server(), [pid], opts :: Keyword.t()) ::
{:ok, pool_ref :: any, module, checkin_time, state :: any}
| {:error, Exception.t()}
def checkout(pool, callers, opts) do
queue? = Keyword.get(opts, :queue, @queue)
now = System.monotonic_time(@time_unit)
timeout = abs_timeout(now, opts)
case checkout(pool, callers, queue?, now, timeout) do
{:ok, _, _, _, _} = ok ->
ok
{:error, %DBConnection.ConnectionError{} = connection_error} = error ->
:telemetry.execute(
[:db_connection, :connection_error],
%{count: 1},
%{
error: connection_error,
opts: opts
}
)
error
{:error, _} = error ->
error
{:redirect, caller, proxy} ->
case checkout(proxy, [caller], opts) do
{:ok, _, _, _, _} = ok ->
ok
{:error, %DBConnection.ConnectionError{message: message} = exception} ->
{:error,
%{
exception
| message:
"could not checkout the connection owned by #{Util.inspect_pid(caller)}. " <>
"When using the sandbox, connections are shared, so this may imply " <>
"another process is using a connection. Reason: #{message}"
}}
{:error, _} = error ->
error
end
{:exit, reason} ->
exit({reason, {__MODULE__, :checkout, [pool, opts]}})
end
end
@spec checkin(pool_ref :: any) :: :ok
def checkin(pool_ref) do
# Note we may call checkin after a disconnect/stop. For this reason, we choose
# to not change the status on checkin but strictly speaking nobody can access
# the holder after disconnect/stop unless they store a copy of %DBConnection{}.
# Note status can't be :aborted as aborted is always reverted at the end of a
# transaction.
done(pool_ref, [{conn(:lock) + 1, nil}], :checkin, System.monotonic_time())
end
@spec disconnect(pool_ref :: any, err :: Exception.t()) :: :ok
def disconnect(pool_ref, err) do
done(pool_ref, [{conn(:status) + 1, :error}], :disconnect, err)
end
@spec stop(pool_ref :: any, err :: Exception.t()) :: :ok
def stop(pool_ref, err) do
done(pool_ref, [{conn(:status) + 1, :error}], :stop, err)
end
@spec handle(pool_ref :: any, fun :: atom, args :: [term], Keyword.t()) :: tuple
def handle(pool_ref, fun, args, opts) do
handle_or_cleanup(:handle, pool_ref, fun, args, opts)
end
@spec cleanup(pool_ref :: any, fun :: atom, args :: [term], Keyword.t()) :: tuple
def cleanup(pool_ref, fun, args, opts) do
handle_or_cleanup(:cleanup, pool_ref, fun, args, opts)
end
defp handle_or_cleanup(type, pool_ref, fun, args, opts) do
pool_ref(holder: holder, lock: lock) = pool_ref
try do
:ets.lookup(holder, :conn)
rescue
ArgumentError ->
msg = "connection is closed because of an error, disconnect or timeout"
{:disconnect, DBConnection.ConnectionError.exception(msg), _state = :unused}
else
[conn(lock: conn_lock)] when conn_lock != lock ->
raise "an outdated connection has been given to DBConnection on #{fun}/#{length(args) + 2}"
[conn(status: :error)] ->
msg = "connection is closed because of an error, disconnect or timeout"
{:disconnect, DBConnection.ConnectionError.exception(msg), _state = :unused}
[conn(status: :aborted)] when type != :cleanup ->
msg = "transaction rolling back"
{:disconnect, DBConnection.ConnectionError.exception(msg), _state = :unused}
[conn(module: module, state: state)] ->
holder_apply(holder, module, fun, args ++ [opts, state])
end
end
## Pool state helpers API (invoked by callers)
@spec put_state(pool_ref :: any, term) :: :ok
def put_state(pool_ref(holder: sink_holder), state) do
:ets.update_element(sink_holder, :conn, [{conn(:state) + 1, state}])
:ok
end
@spec status?(pool_ref :: any, :ok | :aborted) :: boolean()
def status?(pool_ref(holder: holder), status) do
try do
:ets.lookup_element(holder, :conn, conn(:status) + 1) == status
rescue
ArgumentError -> false
end
end
@spec put_status(pool_ref :: any, :ok | :aborted) :: boolean()
def put_status(pool_ref(holder: holder), status) do
try do
:ets.update_element(holder, :conn, [{conn(:status) + 1, status}])
rescue
ArgumentError -> false
end
end
## Pool callbacks (invoked by pools)
@spec reply_redirect({pid, reference}, pid | :shared | :auto, GenServer.server()) :: :ok
def reply_redirect(from, caller, redirect) do
GenServer.reply(from, {:redirect, caller, redirect})
:ok
end
@spec reply_error({pid, reference}, Exception.t()) :: :ok
def reply_error(from, exception) do
GenServer.reply(from, {:error, exception})
:ok
end
@spec handle_checkout(t, {pid, reference}, reference, checkin_time) :: boolean
def handle_checkout(holder, {pid, mref}, ref, checkin_time) do
:ets.give_away(holder, pid, {mref, ref, checkin_time})
rescue
ArgumentError ->
if Process.alive?(pid) or :ets.info(holder, :owner) != self() do
raise ArgumentError, no_holder(holder, pid)
else
false
end
end
@spec handle_deadline(t, reference) :: boolean
def handle_deadline(holder, deadline) do
:ets.lookup_element(holder, :conn, conn(:deadline) + 1)
rescue
ArgumentError -> false
else
^deadline -> true
_ -> false
end
@spec handle_ping(t) :: true
def handle_ping(holder) do
:ets.lookup(holder, :conn)
rescue
ArgumentError ->
raise ArgumentError, no_holder(holder, nil)
else
[conn(connection: conn, state: state)] ->
DBConnection.Connection.ping({conn, holder}, state)
:ets.delete(holder)
true
end
@spec handle_disconnect(t, Exception.t()) :: boolean
def handle_disconnect(holder, err) do
handle_done(holder, &DBConnection.Connection.disconnect/3, err)
end
@spec handle_stop(t, term) :: boolean
def handle_stop(holder, err) do
handle_done(holder, &DBConnection.Connection.stop/3, err)
end
@spec maybe_disconnect(t, integer, non_neg_integer) :: boolean()
def maybe_disconnect(holder, start, interval_ms) do
ts = :ets.lookup_element(holder, :conn, conn(:ts) + 1)
cond do
ts >= start ->
false
interval_ms == 0 ->
true
true ->
pid = :ets.lookup_element(holder, :conn, conn(:connection) + 1)
System.monotonic_time() > hash_pid(pid, interval_ms) + start
end
rescue
_ -> false
else
true ->
opts = [message: "disconnect_all requested", severity: :debug]
handle_disconnect(holder, DBConnection.ConnectionError.exception(opts))
false ->
false
end
## Private
defp checkout(pool, callers, queue?, start, timeout) do
case GenServer.whereis(pool) do
pid when node(pid) == node() ->
checkout_call(pid, callers, queue?, start, timeout)
pid when node(pid) != node() ->
{:exit, {:badnode, node(pid)}}
{_name, node} ->
{:exit, {:badnode, node}}
nil ->
{:exit, :noproc}
end
end
defp checkout_call(pid, callers, queue?, start, timeout) do
lock = Process.monitor(pid)
send(pid, {:db_connection, {self(), lock}, {:checkout, callers, start, queue?}})
receive do
{:"ETS-TRANSFER", holder, pool, {^lock, ref, checkin_time}} ->
Process.demonitor(lock, [:flush])
{deadline, ops} = start_deadline(timeout, pool, ref, holder, start)
:ets.update_element(holder, :conn, [{conn(:lock) + 1, lock} | ops])
pool_ref =
pool_ref(pool: pool, reference: ref, deadline: deadline, holder: holder, lock: lock)
checkout_result(holder, pool_ref, checkin_time)
{^lock, reply} ->
Process.demonitor(lock, [:flush])
reply
{:DOWN, ^lock, _, _, reason} ->
{:exit, reason}
end
end
defp checkout_result(holder, pool_ref, checkin_time) do
try do
:ets.lookup(holder, :conn)
rescue
ArgumentError ->
# Deadline could hit and be handled pool before using connection
msg = "connection not available because deadline reached while in queue"
{:error, DBConnection.ConnectionError.exception(msg)}
else
[conn(module: mod, state: state)] ->
{:ok, pool_ref, mod, checkin_time, state}
end
end
defp no_holder(holder, maybe_pid) do
reason =
case :ets.info(holder, :owner) do
:undefined -> "does not exist"
^maybe_pid -> "is being given to its current owner"
owner when owner != self() -> "does not belong to the giving process"
_ -> "could not be given away"
end
call_reason =
if maybe_pid do
"Error happened when attempting to transfer to #{Util.inspect_pid(maybe_pid)} " <>
"(alive: #{Process.alive?(maybe_pid)})"
else
"Error happened when looking up connection"
end
"""
#{inspect(__MODULE__)} #{inspect(holder)} #{reason}, pool inconsistent.
#{call_reason}.
SELF: #{Util.inspect_pid(self())}
ETS INFO: #{inspect(:ets.info(holder))}
Please report at https://github.com/elixir-ecto/db_connection/issues"
"""
end
defp holder_apply(holder, module, fun, args) do
try do
apply(module, fun, args)
catch
kind, reason ->
{:catch, kind, reason, __STACKTRACE__}
else
result when is_tuple(result) ->
state = :erlang.element(:erlang.tuple_size(result), result)
try do
:ets.update_element(holder, :conn, {conn(:state) + 1, state})
result
rescue
ArgumentError ->
augment_disconnect(result)
end
# If it is not a tuple, we just return it as is so we raise bad return.
result ->
result
end
end
defp augment_disconnect({:disconnect, %DBConnection.ConnectionError{} = err, state}) do
%{message: message} = err
message =
message <>
" (the connection was closed by the pool, " <>
"possibly due to a timeout or because the pool has been terminated)"
{:disconnect, %{err | message: message}, state}
end
defp augment_disconnect(result), do: result
defp done(pool_ref, ops, tag, info) do
pool_ref(pool: pool, reference: ref, deadline: deadline, holder: holder) = pool_ref
cancel_deadline(deadline)
try do
:ets.update_element(holder, :conn, [{conn(:deadline) + 1, nil} | ops])
:ets.give_away(holder, pool, {tag, ref, info})
rescue
ArgumentError -> :ok
else
true -> :ok
end
end
defp handle_done(holder, stop, err) do
:ets.lookup(holder, :conn)
rescue
ArgumentError ->
false
else
[conn(connection: pid, deadline: deadline, state: state)] ->
cancel_deadline(deadline)
:ets.delete(holder)
stop.({pid, holder}, err, state)
true
end
defp abs_timeout(now, opts) do
case Keyword.get(opts, :timeout, @timeout) do
:infinity -> Keyword.get(opts, :deadline)
timeout -> min(now + timeout, Keyword.get(opts, :deadline))
end
end
defp start_deadline(nil, _, _, _, _) do
{nil, []}
end
defp start_deadline(timeout, pid, ref, holder, start) do
deadline =
:erlang.start_timer(timeout, pid, {ref, holder, self(), timeout - start}, abs: true)
{deadline, [{conn(:deadline) + 1, deadline}]}
end
defp cancel_deadline(nil) do
:ok
end
defp cancel_deadline(deadline) do
:erlang.cancel_timer(deadline, async: true, info: false)
end
defp hash_pid(pid, interval_ms) do
hash = :erlang.phash2(pid, interval_ms)
System.convert_time_unit(hash, :millisecond, :native)
end
end

View File

@@ -0,0 +1,83 @@
defmodule DBConnection.LogEntry do
@moduledoc """
Struct containing log entry information.
See `t:t/0` for information on the fields.
"""
defstruct [
:call,
:query,
:params,
:result,
:pool_time,
:connection_time,
:decode_time,
:idle_time
]
@typedoc """
Log entry information.
* `:call` - The `DBConnection` function called
* `:query` - The query used by the function
* `:params` - The params passed to the function (if any)
* `:result` - The result of the call
* `:pool_time` - The length of time awaiting a connection from the pool (if
the connection was not already checked out)
* `:connection_time` - The length of time using the connection (if a
connection was used)
* `:decode_time` - The length of time decoding the result (if decoded the
result using `DBConnection.Query.decode/3`)
* `:idle_time` - The amount of time the connection was idle before use
All times are in the native time units of the VM, see
`System.monotonic_time/0`.
"""
@type t :: %__MODULE__{
call: atom,
query: any,
params: any,
result: {:ok, any} | {:ok, any, any} | {:error, Exception.t()},
pool_time: non_neg_integer | nil,
connection_time: non_neg_integer | nil,
idle_time: non_neg_integer | nil,
decode_time: non_neg_integer | nil
}
@doc false
def new(call, query, params, times, result) do
entry = %__MODULE__{call: call, query: query, params: params, result: result}
parse_times(times, entry)
end
## Helpers
defp parse_times([], entry), do: entry
defp parse_times(times, entry) do
stop = :erlang.monotonic_time()
{_, entry} = Enum.reduce(times, {stop, entry}, &parse_time/2)
entry
end
defp parse_time({:decode, start}, {stop, entry}) do
{start, %{entry | decode_time: stop - start}}
end
defp parse_time({:checkout, start}, {stop, entry}) do
{start, %{entry | pool_time: stop - start}}
end
defp parse_time({:checkin, start}, {stop, entry}) do
# The checkin time was most likely before checkout but it is
# not guaranteed as they are tracked by different processes.
# There should be no further measurements after checkin.
{stop, %{entry | idle_time: max(stop - start, 0)}}
end
defp parse_time({_, start}, {stop, entry}) do
%{connection_time: connection_time} = entry
{start, %{entry | connection_time: (connection_time || 0) + (stop - start)}}
end
end

View File

@@ -0,0 +1,155 @@
defmodule DBConnection.OwnershipError do
@moduledoc """
An exception for when errors with ownership occur.
"""
defexception [:message]
def exception(message), do: %DBConnection.OwnershipError{message: message}
end
defmodule DBConnection.Ownership do
@moduledoc """
A DBConnection pool that requires explicit checkout and checkin
as a mechanism to coordinate between processes.
## Options
* `:ownership_mode` - When mode is `:manual`, all connections must
be explicitly checked out before by using `ownership_checkout/2`.
Otherwise, mode is `:auto` and connections are checked out
implicitly. `{:shared, owner}` mode is also supported so
processes are allowed on demand. On all cases, checkins are
explicit via `ownership_checkin/2`. Defaults to `:auto`.
* `:ownership_timeout` - The maximum time (in milliseconds) that a process
is allowed to own a connection or `:infinity`, default `120_000`.
This timeout exists mostly for sanity checking purposes and can be increased
at will, since DBConnection automatically checks in connections whenever
there is a mode change.
* `:ownership_log` - The `Logger.level` to log ownership changes, or `nil`
not to log, default `nil`.
There are also two experimental options, `:post_checkout` and `:pre_checkin`
which allows a developer to configure what happens when a connection is
checked out and checked in. Those options are meant to be used during tests,
and have the following behaviour:
* `:post_checkout` - it must be an anonymous function that receives the
connection module, the connection state and it must return either
`{:ok, connection_module, connection_state}` or
`{:disconnect, err, connection_module, connection_state}`. This allows
the developer to change the connection module on post checkout. However,
in case of disconnects, the return `connection_module` must be the same
as the `connection_module` given. Defaults to simply returning the given
connection module and state.
* `:pre_checkin` - it must be an anonymous function that receives the
checkin reason (`:checkin`, `{:disconnect, err}` or `{:stop, err}`),
the connection module and the connection state returned by `post_checkout`.
It must return either `{:ok, connection_module, connection_state}` or
`{:disconnect, err, connection_module, connection_state}` where the connection
module is the module given to `:post_checkout` Defaults to simply returning
the given connection module and state.
## Callers lookup
When checking out, the ownership pool first looks if there is a connection
assigned to the current process and then checks if there is a connection
assigned to any of the processes listed under the `$callers` process
dictionary entry. The `$callers` entry is set by default for tasks from
Elixir v1.8.
You can also pass the `:caller` option on checkout with a pid and that
pid will be looked up first, instead of `self()`, and then we fall back
to `$callers`.
"""
alias DBConnection.Ownership.Manager
alias DBConnection.Holder
@behaviour DBConnection.Pool
@doc false
defdelegate child_spec(args), to: Manager
@doc false
@impl DBConnection.Pool
defdelegate disconnect_all(pool, interval, opts), to: Manager
@doc false
@impl DBConnection.Pool
def checkout(pool, callers, opts) do
case Manager.proxy_for(callers, opts) do
{caller, pool} -> Holder.checkout(pool, [caller], opts)
nil -> Holder.checkout(pool, callers, opts)
end
end
@doc false
@impl DBConnection.Pool
defdelegate get_connection_metrics(pool), to: Manager
@doc """
Explicitly checks a connection out from the ownership manager.
It may return `:ok` if the connection is checked out.
`{:already, :owner | :allowed}` if the caller process already
has a connection, or raise if there was an error.
"""
@spec ownership_checkout(GenServer.server(), Keyword.t()) ::
:ok | {:already, :owner | :allowed}
def ownership_checkout(manager, opts) do
with {:ok, pid} <- Manager.checkout(manager, opts) do
case Holder.checkout(pid, [self()], opts) do
{:ok, pool_ref, _module, _idle_time, _state} ->
Holder.checkin(pool_ref)
{:error, err} ->
raise err
end
end
end
@doc """
Changes the ownership mode.
`mode` may be `:auto`, `:manual` or `{:shared, owner}`.
The operation will always succeed when setting the mode to
`:auto` or `:manual`. It may fail with reason `:not_owner`
or `:not_found` when setting `{:shared, pid}` and the
given pid does not own any connection. May return
`:already_shared` if another process set the ownership
mode to `{:shared, _}` and is still alive.
"""
@spec ownership_mode(GenServer.server(), :auto | :manual | {:shared, pid}, Keyword.t()) ::
:ok | :already_shared | :not_owner | :not_found
defdelegate ownership_mode(manager, mode, opts), to: Manager, as: :mode
@doc """
Checks a connection back in.
A connection can only be checked back in by its owner.
"""
@spec ownership_checkin(GenServer.server(), Keyword.t()) ::
:ok | :not_owner | :not_found
defdelegate ownership_checkin(manager, opts), to: Manager, as: :checkin
@doc """
Allows the process given by `allow` to use the connection checked out
by `owner_or_allowed`.
It may return `:ok` if the connection is checked out.
`{:already, :owner | :allowed}` if the `allow` process already
has a connection. `owner_or_allowed` may either be the owner or any
other allowed process. Returns `:not_found` if the given process
does not have any connection checked out.
Setting the `unallow_existing` option to `true` will remove the process given by `allow` from
any existing allowance it may have (this is necessary because a given process can only be
allowed on a single connection at a time).
"""
@spec ownership_allow(GenServer.server(), owner_or_allowed :: pid, allow :: pid, Keyword.t()) ::
:ok | {:already, :owner | :allowed} | :not_found
defdelegate ownership_allow(manager, owner, allow, opts), to: Manager, as: :allow
end

View File

@@ -0,0 +1,439 @@
defmodule DBConnection.Ownership.Manager do
@moduledoc false
use GenServer
require Logger
alias DBConnection.Ownership.Proxy
alias DBConnection.Util
@timeout 5_000
@callback start_link({module, opts :: Keyword.t()}) ::
GenServer.on_start()
def start_link({module, opts}) do
{owner_opts, pool_opts} = Keyword.split(opts, [:name])
GenServer.start_link(__MODULE__, {module, owner_opts, pool_opts}, owner_opts)
end
@callback disconnect_all(GenServer.server(), non_neg_integer, Keyword.t()) :: :ok
def disconnect_all(pool, interval, opts) do
inner_pool = GenServer.call(pool, :pool, :infinity)
DBConnection.ConnectionPool.disconnect_all(inner_pool, interval, opts)
end
@spec proxy_for(callers :: [pid], Keyword.t()) :: {caller :: pid, proxy :: pid} | nil
def proxy_for(callers, opts) do
case Keyword.fetch(opts, :name) do
{:ok, name} ->
Enum.find_value(callers, &List.first(:ets.lookup(name, &1)))
:error ->
nil
end
end
@spec checkout(GenServer.server(), Keyword.t()) ::
{:ok, pid} | {:already, :owner | :allowed}
def checkout(manager, opts) do
GenServer.call(manager, {:checkout, opts}, :infinity)
end
@spec checkin(GenServer.server(), Keyword.t()) ::
:ok | :not_owner | :not_found
def checkin(manager, opts) do
timeout = Keyword.get(opts, :timeout, @timeout)
GenServer.call(manager, :checkin, timeout)
end
@spec mode(GenServer.server(), :auto | :manual | {:shared, pid}, Keyword.t()) ::
:ok | :already_shared | :not_owner | :not_found
def mode(manager, mode, opts)
when mode in [:auto, :manual]
when elem(mode, 0) == :shared and is_pid(elem(mode, 1)) do
timeout = Keyword.get(opts, :timeout, @timeout)
GenServer.call(manager, {:mode, mode}, timeout)
end
@spec allow(GenServer.server(), parent :: pid, allow :: pid, Keyword.t()) ::
:ok | {:already, :owner | :allowed} | :not_found
def allow(manager, parent, allow, opts) do
timeout = Keyword.get(opts, :timeout, @timeout)
passed_opts = Keyword.take(opts, [:unallow_existing])
GenServer.call(manager, {:allow, parent, allow, passed_opts}, timeout)
end
@spec get_connection_metrics(GenServer.server()) ::
{:ok, [DBConnection.Pool.connection_metrics()]} | :error
def get_connection_metrics(manager) do
GenServer.call(manager, :get_connection_metrics, :infinity)
end
## Callbacks
@impl true
def init({module, owner_opts, pool_opts}) do
DBConnection.register_as_pool(module)
ets =
case Keyword.fetch(owner_opts, :name) do
{:ok, name} when is_atom(name) ->
:ets.new(name, [
:set,
:named_table,
:protected,
read_concurrency: true,
decentralized_counters: true
])
_ ->
nil
end
# We can only start the connection pool directly because
# neither the pool's GenServer nor the manager trap exits.
# Otherwise we would need a supervisor plus a watcher process.
pool_opts = Keyword.delete(pool_opts, :pool)
{:ok, pool} = DBConnection.start_link(module, pool_opts)
log = Keyword.get(pool_opts, :ownership_log, nil)
mode = Keyword.get(pool_opts, :ownership_mode, :auto)
checkout_opts = Keyword.take(pool_opts, [:ownership_timeout, :queue_target, :queue_interval])
{:ok,
%{
pool: pool,
checkouts: %{},
owners: %{},
checkout_opts: checkout_opts,
mode: mode,
mode_ref: nil,
ets: ets,
log: log
}}
end
@impl true
def handle_call(:get_connection_metrics, _from, %{pool: pool, owners: owners, log: log} = state) do
pool_metrics = DBConnection.ConnectionPool.get_connection_metrics(pool)
proxy_metrics =
owners
|> Enum.map(fn {_, {proxy, _, _}} ->
try do
GenServer.call(proxy, :get_connection_metrics)
catch
:exit, reason ->
if log do
Logger.log(
log,
"Caught :exit while calling :get_connection_metrics due to #{inspect(reason)}"
)
end
nil
end
end)
|> Enum.reject(&is_nil/1)
{:reply, pool_metrics ++ proxy_metrics, state}
end
def handle_call(:pool, _from, %{pool: pool} = state) do
{:reply, pool, state}
end
def handle_call({:mode, {:shared, shared}}, {caller, _}, %{mode: {:shared, current}} = state) do
cond do
shared == current ->
{:reply, :ok, state}
Process.alive?(current) ->
{:reply, :already_shared, state}
true ->
share_and_reply(state, shared, caller)
end
end
def handle_call({:mode, {:shared, shared}}, {caller, _}, state) do
share_and_reply(state, shared, caller)
end
def handle_call({:mode, mode}, _from, %{mode: mode} = state) do
{:reply, :ok, state}
end
def handle_call({:mode, mode}, {caller, _}, state) do
state = proxy_checkin_all_except(state, [], caller)
{:reply, :ok, %{state | mode: mode, mode_ref: nil}}
end
def handle_call(:checkin, {caller, _}, state) do
{reply, state} = proxy_checkin(state, caller, caller)
{:reply, reply, state}
end
def handle_call({:allow, caller, allow, opts}, _from, %{checkouts: checkouts} = state) do
unallow_existing = Keyword.get(opts, :unallow_existing, false)
kind = already_checked_out(checkouts, allow)
if !unallow_existing && kind do
{:reply, {:already, kind}, state}
else
case Map.get(checkouts, caller, :not_found) do
{:owner, ref, proxy} ->
state =
if unallow_existing, do: owner_unallow(state, caller, allow, ref, proxy), else: state
{:reply, :ok, owner_allow(state, caller, allow, ref, proxy)}
{:allowed, ref, proxy} ->
state =
if unallow_existing, do: owner_unallow(state, caller, allow, ref, proxy), else: state
{:reply, :ok, owner_allow(state, caller, allow, ref, proxy)}
:not_found ->
{:reply, :not_found, state}
end
end
end
def handle_call({:checkout, opts}, {caller, _}, %{checkouts: checkouts} = state) do
if kind = already_checked_out(checkouts, caller) do
{:reply, {:already, kind}, state}
else
{proxy, state} = proxy_checkout(state, caller, opts)
{:reply, {:ok, proxy}, state}
end
end
@impl true
def handle_info({:db_connection, from, {:checkout, callers, _now, queue?}}, state) do
%{checkouts: checkouts, mode: mode, checkout_opts: checkout_opts} = state
caller = find_caller(callers, checkouts, mode)
case Map.get(checkouts, caller, :not_found) do
{status, _ref, proxy} when status in [:owner, :allowed] ->
DBConnection.Holder.reply_redirect(from, caller, proxy)
{:noreply, state}
:not_found when mode == :auto ->
{proxy, state} = proxy_checkout(state, caller, [queue: queue?] ++ checkout_opts)
DBConnection.Holder.reply_redirect(from, caller, proxy)
{:noreply, state}
:not_found when mode == :manual ->
not_found(from, mode)
{:noreply, state}
:not_found ->
{:shared, shared} = mode
{:owner, _ref, proxy} = Map.fetch!(checkouts, shared)
DBConnection.Holder.reply_redirect(from, shared, proxy)
{:noreply, state}
end
end
def handle_info({:DOWN, ref, _, _, _}, state) do
{:noreply, state |> owner_down(ref) |> unshare(ref)}
end
def handle_info(_msg, state) do
{:noreply, state}
end
defp already_checked_out(checkouts, pid) do
case Map.get(checkouts, pid, :not_found) do
{:owner, _, _} -> :owner
{:allowed, _, _} -> :allowed
:not_found -> nil
end
end
defp proxy_checkout(state, caller, opts) do
%{pool: pool, checkouts: checkouts, owners: owners, ets: ets, log: log, mode: mode} = state
{:ok, proxy} =
DynamicSupervisor.start_child(
DBConnection.Ownership.Supervisor,
{DBConnection.Ownership.Proxy, {caller, pool, opts}}
)
if log do
Logger.log(log, fn ->
[
Util.inspect_pid(caller),
" checked out connection in ",
inspect(mode),
" mode using proxy ",
Util.inspect_pid(proxy)
]
end)
end
ref = Process.monitor(proxy)
checkouts = Map.put(checkouts, caller, {:owner, ref, proxy})
owners = Map.put(owners, ref, {proxy, caller, []})
ets && :ets.insert(ets, {caller, proxy})
{proxy, %{state | checkouts: checkouts, owners: owners}}
end
defp proxy_checkin(state, maybe_owner, caller) do
case get_and_update_in(state.checkouts, &Map.pop(&1, maybe_owner, :not_found)) do
{{:owner, ref, proxy}, state} ->
Proxy.stop(proxy, caller)
{:ok, state |> owner_down(ref) |> unshare(ref)}
{{:allowed, _, _}, _} ->
{:not_owner, state}
{:not_found, _} ->
{:not_found, state}
end
end
defp proxy_checkin_all_except(state, except, caller) do
Enum.reduce(state.checkouts, state, fn {pid, _}, state ->
if pid in except do
state
else
{_, state} = proxy_checkin(state, pid, caller)
state
end
end)
end
defp owner_allow(%{ets: ets, log: log} = state, caller, allow, ref, proxy) do
if log do
Logger.log(log, fn ->
[
Util.inspect_pid(allow),
" was allowed by ",
Util.inspect_pid(caller),
" on proxy ",
Util.inspect_pid(proxy)
]
end)
end
state = put_in(state.checkouts[allow], {:allowed, ref, proxy})
state =
update_in(state.owners[ref], fn {proxy, caller, allowed} ->
{proxy, caller, [allow | List.delete(allowed, allow)]}
end)
ets && :ets.insert(ets, {allow, proxy})
state
end
defp owner_unallow(%{ets: ets, log: log} = state, caller, unallow, ref, proxy) do
if log do
Logger.log(log, fn ->
[
Util.inspect_pid(unallow),
" was unallowed by ",
Util.inspect_pid(caller),
" on proxy ",
Util.inspect_pid(proxy)
]
end)
end
state = update_in(state.checkouts, &Map.delete(&1, unallow))
state =
update_in(state.owners[ref], fn {proxy, caller, allowed} ->
{proxy, caller, List.delete(allowed, unallow)}
end)
ets && :ets.delete(ets, {unallow, proxy})
state
end
defp owner_down(%{ets: ets, log: log} = state, ref) do
case get_and_update_in(state.owners, &Map.pop(&1, ref)) do
{{proxy, caller, allowed}, state} ->
Process.demonitor(ref, [:flush])
entries = [caller | allowed]
if log do
Logger.log(log, fn ->
[
Enum.map_join(entries, ", ", &Util.inspect_pid/1),
" lost connection from proxy ",
Util.inspect_pid(proxy)
]
end)
end
ets && Enum.each(entries, &:ets.delete(ets, &1))
update_in(state.checkouts, &Map.drop(&1, entries))
{nil, state} ->
state
end
end
defp share_and_reply(%{checkouts: checkouts} = state, shared, caller) do
case Map.get(checkouts, shared, :not_found) do
{:owner, ref, _} ->
state = proxy_checkin_all_except(state, [shared], caller)
{:reply, :ok, %{state | mode: {:shared, shared}, mode_ref: ref}}
{:allowed, _, _} ->
{:reply, :not_owner, state}
:not_found ->
{:reply, :not_found, state}
end
end
defp unshare(%{mode_ref: ref} = state, ref) do
%{state | mode: :manual, mode_ref: nil}
end
defp unshare(state, _ref) do
state
end
defp find_caller(callers, checkouts, :manual) do
Enum.find(callers, &Map.has_key?(checkouts, &1)) || hd(callers)
end
defp find_caller([caller | _], _checkouts, _mode) do
caller
end
defp not_found({pid, _} = from, mode) do
msg = """
cannot find ownership process for #{Util.inspect_pid(pid)}
using mode #{inspect(mode)}.
When using ownership, you must manage connections in one
of the four ways:
* By explicitly checking out a connection
* By explicitly allowing a spawned process
* By running the pool in shared mode
* By using :caller option with allowed process
The first two options require every new process to explicitly
check a connection out or be allowed by calling checkout or
allow respectively.
The third option requires a {:shared, pid} mode to be set.
If using shared mode in tests, make sure your tests are not
async.
The fourth option requires [caller: pid] to be used when
checking out a connection from the pool. The caller process
should already be allowed on a connection.
If you are reading this error, it means you have not done one
of the steps above or that the owner process has crashed.
"""
DBConnection.Holder.reply_error(from, DBConnection.OwnershipError.exception(msg))
end
end

View File

@@ -0,0 +1,334 @@
defmodule DBConnection.Ownership.Proxy do
@moduledoc false
alias DBConnection.Holder
alias DBConnection.Util
use GenServer, restart: :temporary
@time_unit 1000
@ownership_timeout 120_000
@queue_target 50
@queue_interval 1000
def start_link({caller, pool, pool_opts}) do
GenServer.start_link(__MODULE__, {caller, pool, pool_opts}, [])
end
def stop(proxy, caller) do
GenServer.cast(proxy, {:stop, caller})
end
# Callbacks
@impl true
def init({caller, pool, pool_opts}) do
pool_opts =
pool_opts
|> Keyword.put(:timeout, :infinity)
|> Keyword.delete(:deadline)
owner_ref = Process.monitor(caller)
ownership_timeout = Keyword.get(pool_opts, :ownership_timeout, @ownership_timeout)
timeout = Keyword.get(pool_opts, :queue_target, @queue_target) * 2
interval = Keyword.get(pool_opts, :queue_interval, @queue_interval)
pre_checkin = Keyword.get(pool_opts, :pre_checkin, fn _, mod, state -> {:ok, mod, state} end)
post_checkout = Keyword.get(pool_opts, :post_checkout, &{:ok, &1, &2})
state = %{
client: nil,
timer: nil,
holder: nil,
timeout: timeout,
interval: interval,
poll: nil,
owner: {caller, owner_ref},
pool: pool,
pool_ref: nil,
pool_opts: pool_opts,
queue: :queue.new(),
mod: nil,
pre_checkin: pre_checkin,
post_checkout: post_checkout,
ownership_timer: start_timer(caller, ownership_timeout)
}
now = System.monotonic_time(@time_unit)
{:ok, start_poll(now, state)}
end
@impl true
def handle_info({:DOWN, ref, _, pid, _reason}, %{owner: {_, ref}} = state) do
shutdown("owner #{Util.inspect_pid(pid)} exited", state)
end
def handle_info({:timeout, deadline, {_ref, holder, pid, len}}, %{holder: holder} = state) do
if Holder.handle_deadline(holder, deadline) do
message =
"client #{Util.inspect_pid(pid)} timed out because " <>
"it queued and checked out the connection for longer than #{len}ms"
shutdown(message, state)
else
{:noreply, state}
end
end
def handle_info(
{:timeout, timer, {__MODULE__, pid, timeout}},
%{ownership_timer: timer} = state
) do
message =
"owner #{Util.inspect_pid(pid)} timed out because " <>
"it owned the connection for longer than #{timeout}ms (set via the :ownership_timeout option)"
# We don't invoke shutdown because this is always a disconnect, even if there is no client.
# On the other hand, those timeouts are unlikely to trigger, as it defaults to 2 mins.
pool_disconnect(DBConnection.ConnectionError.exception(message), false, state)
end
def handle_info({:timeout, poll, time}, %{poll: poll} = state) do
state = timeout(time, state)
{:noreply, start_poll(time, state)}
end
def handle_info(
{:db_connection, from, {:checkout, _caller, _now, _queue?}},
%{holder: nil} = state
) do
%{pool: pool, pool_opts: pool_opts, owner: {_, owner_ref}, post_checkout: post_checkout} =
state
case Holder.checkout(pool, [self()], pool_opts) do
{:ok, pool_ref, original_mod, _idle_time, conn_state} ->
case post_checkout.(original_mod, conn_state) do
{:ok, conn_mod, conn_state} ->
holder = Holder.new(self(), owner_ref, conn_mod, conn_state)
state = %{state | pool_ref: pool_ref, holder: holder, mod: original_mod}
checkout(from, state)
{:disconnect, err, ^original_mod, _conn_state} ->
Holder.disconnect(pool_ref, err)
Holder.reply_error(from, err)
{:stop, {:shutdown, err}, state}
end
{:error, err} ->
Holder.reply_error(from, err)
{:stop, {:shutdown, err}, state}
end
end
def handle_info(
{:db_connection, from, {:checkout, _caller, _now, _queue?}},
%{client: nil} = state
) do
checkout(from, state)
end
def handle_info({:db_connection, from, {:checkout, _caller, now, queue?}}, state) do
if queue? do
%{queue: queue} = state
queue = :queue.in({now, from}, queue)
{:noreply, %{state | queue: queue}}
else
message = "connection not available and queuing is disabled"
err = DBConnection.ConnectionError.exception(message)
Holder.reply_error(from, err)
{:noreply, state}
end
end
def handle_info(
{:"ETS-TRANSFER", holder, _, {msg, ref, extra}},
%{holder: holder, client: {_, ref, _}} = state
) do
case msg do
:checkin -> checkin(state)
:disconnect -> pool_disconnect(extra, true, state)
:stop -> pool_stop(extra, state)
end
end
def handle_info({:"ETS-TRANSFER", holder, pid, ref}, %{holder: holder, owner: {_, ref}} = state) do
shutdown("client #{Util.inspect_pid(pid)} exited", state)
end
@impl true
def handle_cast({:stop, caller}, %{owner: {owner, _}} = state) do
message =
"#{Util.inspect_pid(caller)} checked in the connection owned by #{Util.inspect_pid(owner)}"
message =
case pruned_stacktrace(caller) do
[] ->
message
current_stack ->
message <>
"\n\n#{Util.inspect_pid(caller)} triggered the checkin at location:\n\n" <>
Exception.format_stacktrace(current_stack)
end
shutdown(message, state)
end
@impl true
def handle_call(
:get_connection_metrics,
_,
%{queue: queue, holder: holder, client: client} = state
) do
connection_metrics = %{
source: {:proxy, self()},
ready_conn_count:
if is_nil(holder) or not is_nil(client) do
0
else
1
end,
checkout_queue_length: :queue.len(queue)
}
{:reply, connection_metrics, state}
end
defp checkout({pid, ref} = from, %{holder: holder} = state) do
if Holder.handle_checkout(holder, from, ref, nil) do
{:noreply, %{state | client: {pid, ref, pruned_stacktrace(pid)}}}
else
next(state)
end
end
defp checkin(state) do
next(%{state | client: nil})
end
defp next(%{queue: queue} = state) do
case :queue.out(queue) do
{{:value, {_, from}}, queue} ->
checkout(from, %{state | queue: queue})
{:empty, queue} ->
{:noreply, %{state | queue: queue}}
end
end
defp start_timer(_, :infinity), do: nil
defp start_timer(pid, timeout) do
:erlang.start_timer(timeout, self(), {__MODULE__, pid, timeout})
end
# If shutting down but it has no client, checkin
defp shutdown(reason, %{client: nil} = state) do
pool_checkin(reason, state)
end
# If shutting down but it has a client, disconnect
defp shutdown(reason, %{client: {client, _, checkout_stack}} = state) do
reason =
case pruned_stacktrace(client) do
[] ->
reason
current_stack ->
reason <>
"""
\n\nClient #{Util.inspect_pid(client)} is still using a connection from owner at location:
#{Exception.format_stacktrace(current_stack)}
The connection itself was checked out by #{Util.inspect_pid(client)} at location:
#{Exception.format_stacktrace(checkout_stack)}
"""
end
err = DBConnection.ConnectionError.exception(reason)
pool_disconnect(err, false, state)
end
## Helpers
defp pool_checkin(reason, state) do
checkin = fn pool_ref, _ -> Holder.checkin(pool_ref) end
pool_done(reason, state, :checkin, false, checkin, &Holder.disconnect/2)
end
defp pool_disconnect(err, keep_alive?, state) do
disconnect = &Holder.disconnect/2
pool_done(err, state, {:disconnect, err}, keep_alive?, disconnect, disconnect)
end
defp pool_stop(err, state) do
stop = &Holder.stop/2
pool_done(err, state, {:stop, err}, false, stop, stop)
end
defp pool_done(err, state, op, keep_alive?, done, stop_or_disconnect) do
%{holder: holder, pool_ref: pool_ref, pre_checkin: pre_checkin, mod: original_mod} = state
if holder do
{conn_mod, conn_state} = Holder.delete(holder)
case pre_checkin.(op, conn_mod, conn_state) do
{:ok, ^original_mod, conn_state} ->
Holder.put_state(pool_ref, conn_state)
done.(pool_ref, err)
if keep_alive? do
{:noreply, %{state | holder: nil}}
else
{:stop, {:shutdown, err}, state}
end
{:disconnect, err, ^original_mod, conn_state} ->
Holder.put_state(pool_ref, conn_state)
stop_or_disconnect.(pool_ref, err)
{:stop, {:shutdown, err}, state}
end
else
{:stop, {:shutdown, err}, state}
end
end
defp start_poll(now, %{interval: interval} = state) do
timeout = now + interval
poll = :erlang.start_timer(timeout, self(), timeout, abs: true)
%{state | poll: poll}
end
defp timeout(time, %{queue: queue, timeout: timeout} = state) do
case :queue.out(queue) do
{{:value, {sent, from}}, queue} when sent + timeout < time ->
drop(time - sent, from)
timeout(time, %{state | queue: queue})
{_, _} ->
state
end
end
defp drop(delay, from) do
message =
"connection not available and request was dropped from queue after #{delay}ms. " <>
"You can configure how long requests wait in the queue using :queue_target and " <>
":queue_interval. See DBConnection.start_link/2 for more information"
err = DBConnection.ConnectionError.exception(message, :queue_timeout)
Holder.reply_error(from, err)
end
@prune_modules [:gen, GenServer, DBConnection, DBConnection.Holder, DBConnection.Ownership]
defp pruned_stacktrace(pid) do
case Process.info(pid, :current_stacktrace) do
{:current_stacktrace, stacktrace} ->
Enum.drop_while(stacktrace, &match?({mod, _, _, _} when mod in @prune_modules, &1))
_ ->
[]
end
end
end

View File

@@ -0,0 +1,19 @@
defmodule DBConnection.Pool do
@moduledoc false
@type pool :: GenServer.server()
@type connection_metrics :: %{
source: {:pool | :proxy, pid()},
ready_conn_count: non_neg_integer(),
checkout_queue_length: non_neg_integer()
}
@callback disconnect_all(pool, interval :: term, options :: keyword) :: :ok
@callback checkout(pool, callers :: [pid], options :: keyword) ::
{:ok, pool_ref :: term, module, checkin_time :: non_neg_integer() | nil,
state :: term}
| {:error, Exception.t()}
@callback get_connection_metrics(pool :: pool()) :: [connection_metrics()]
end

View File

@@ -0,0 +1,57 @@
defprotocol DBConnection.Query do
@moduledoc """
The `DBConnection.Query` protocol is responsible for preparing and
encoding queries.
All `DBConnection.Query` functions are executed in the caller process which
means it's safe to, for example, raise exceptions or do blocking calls as
they won't affect the connection process.
"""
@doc """
Parse a query.
This function is called to parse a query term before it is prepared using a
connection callback module.
See `DBConnection.prepare/3`.
"""
@spec parse(any, Keyword.t()) :: any
def parse(query, opts)
@doc """
Describe a query.
This function is called to describe a query after it is prepared using a
connection callback module.
See `DBConnection.prepare/3`.
"""
@spec describe(any, Keyword.t()) :: any
def describe(query, opts)
@doc """
Encode parameters using a query.
This function is called to encode a query before it is executed using a
connection callback module.
If this function raises `DBConnection.EncodeError`, then the query is
prepared once again.
See `DBConnection.execute/3`.
"""
@spec encode(any, any, Keyword.t()) :: any
def encode(query, params, opts)
@doc """
Decode a result using a query.
This function is called to decode a result after it is returned by a
connection callback module.
See `DBConnection.execute/3`.
"""
@spec decode(any, any, Keyword.t()) :: any
def decode(query, result, opts)
end

View File

@@ -0,0 +1,47 @@
defmodule DBConnection.Task do
@moduledoc false
@name __MODULE__
require DBConnection.Holder
def run_child(mod, state, fun, opts) do
arg = [fun, self(), opts]
{:ok, pid} = Task.Supervisor.start_child(@name, __MODULE__, :init, arg)
ref = Process.monitor(pid)
_ = DBConnection.Holder.update(pid, ref, mod, state)
{pid, ref}
end
def init(fun, parent, opts) do
try do
Process.link(parent)
catch
:error, :noproc ->
exit({:shutdown, :noproc})
end
receive do
{:"ETS-TRANSFER", holder, ^parent, {:checkin, ref, _extra}} ->
Process.unlink(parent)
pool_ref = DBConnection.Holder.pool_ref(pool: parent, reference: ref, holder: holder)
checkout = {:via, __MODULE__, pool_ref}
_ = DBConnection.run(checkout, make_fun(fun), [pool: __MODULE__] ++ opts)
exit(:normal)
end
end
def checkout({:via, __MODULE__, pool_ref}, _callers, _opts) do
{:ok, pool_ref, _mod = :unused, _idle_time = nil, _state = :unused}
end
defp make_fun(fun) when is_function(fun, 1) do
fun
end
defp make_fun(mfargs) do
fn conn ->
{mod, fun, args} = mfargs
apply(mod, fun, [conn | args])
end
end
end

View File

@@ -0,0 +1,116 @@
defmodule DBConnection.TelemetryListener do
@moduledoc """
A connection listener that emits telemetry events for connection and disconnection
It monitors connection processes and ensures that disconnection events are
always emitted.
## Usage
Start the listener, and pass it under the `:connection_listeners` option when
starting DBConnection:
{:ok, pid} = DBConnection.TelemetryListener.start_link()
{:ok, _conn} = DBConnection.start_link(SomeModule, connection_listeners: [pid])
# Using a tag, which will be sent in telemetry metadata
{:ok, _conn} = DBConnection.start_link(SomeModule, connection_listeners: {[pid], :my_tag})
# Or, with a Supervisor:
Supervisor.start_link([
{DBConnection.TelemetryListener, name: MyListener},
DBConnection.child_spec(SomeModule, connection_listeners: {[MyListener], :my_tag})
])
When using with Ecto, you can pass the `connection_listeners` option to Ecto, and we
recommend passing the repository as the tag. In your supervision tree:
Supervisor.start_link([
{DBConnection.TelemetryListener, name: MyApp.DBListener},
{MyApp.Repo, connection_listeners: {[MyApp.DBListener], MyApp.Repo})
])
## Telemetry events
### Connected
`[:db_connection, :connected]` - Executed after a connection is established.
#### Measurements
* `:count` - Always 1
#### Metadata
* `:pid` - The connection pid
* `:tag` - The connection pool tag
### Disconnected
`[:db_connection, :disconnected]` - Executed after a disconnect.
#### Measurements
* `:count` - Always 1
#### Metadata
* `:pid` - The connection pid
* `:tag` - The connection pool tag
"""
use GenServer
@doc "Starts a telemetry listener"
@spec start_link(GenServer.options()) :: {:ok, pid()}
def start_link(opts \\ []) do
GenServer.start_link(__MODULE__, nil, opts)
end
@impl GenServer
def init(nil) do
{:ok, %{monitoring: %{}}}
end
@impl GenServer
def handle_info({:connected, pid, tag}, state) do
handle_connected(pid, tag, state)
end
def handle_info({:connected, pid}, state) do
handle_connected(pid, nil, state)
end
def handle_info({:disconnected, pid, _}, state) do
handle_disconnected(pid, state)
end
def handle_info({:disconnected, pid}, state) do
handle_disconnected(pid, state)
end
def handle_info({:DOWN, _ref, :process, pid, _reason}, state) do
handle_disconnected(pid, state)
end
defp handle_connected(pid, tag, state) do
:telemetry.execute([:db_connection, :connected], %{count: 1}, %{tag: tag, pid: pid})
ref = Process.monitor(pid)
{:noreply, put_in(state.monitoring[pid], {ref, tag})}
end
defp handle_disconnected(pid, state) do
case state.monitoring[pid] do
# Already handled. We may receive two messages: one from monitor and one
# from listener. For this reason, we need to handle both.
nil ->
{:noreply, state}
{ref, tag} ->
Process.demonitor(ref, [:flush])
:telemetry.execute([:db_connection, :disconnected], %{count: 1}, %{tag: tag, pid: pid})
{:noreply, %{state | monitoring: Map.delete(state.monitoring, pid)}}
end
end
end

View File

@@ -0,0 +1,62 @@
defmodule DBConnection.Util do
@moduledoc false
@doc """
Inspect a pid, including the process label if possible.
"""
def inspect_pid(pid) when is_pid(pid) do
with :undefined <- get_label(pid),
:undefined <- get_name(pid),
:undefined <- get_initial_call(pid) do
inspect(pid)
else
label_or_name_or_call -> "#{inspect(pid)} (#{inspect(label_or_name_or_call)})"
end
end
def inspect_pid(other), do: inspect(other)
defp get_name(pid) do
try do
Process.info(pid, :registered_name)
rescue
_ -> :undefined
else
{:registered_name, name} when is_atom(name) -> name
_ -> :undefined
end
end
@doc """
Set a process label if `Process.set_label/1` is available.
"""
def set_label(label) do
if function_exported?(Process, :set_label, 1) do
Process.set_label(label)
else
:ok
end
end
# Get a process label if `:proc_lib.get_label/1` is available.
defp get_label(pid) do
if function_exported?(:proc_lib, :get_label, 1) do
# Avoid a compiler warning if the function isn't
# defined in your version of Erlang/OTP
apply(:proc_lib, :get_label, [pid])
else
# mimic return value of
# `:proc_lib.get_label/1` when none is set.
# Don't resort to using `Process.info(pid, :dictionary)`,
# as this is not efficient.
:undefined
end
end
defp get_initial_call(pid) do
case Process.info(pid, :initial_call) do
{:initial_call, {mod, _, _}} -> mod
_ -> :undefined
end
end
end

View File

@@ -0,0 +1,65 @@
defmodule DBConnection.Watcher do
@moduledoc false
@name __MODULE__
use GenServer
def start_link(_) do
GenServer.start_link(__MODULE__, :ok, name: @name)
end
def watch(supervisor, args) do
GenServer.call(@name, {:watch, supervisor, args}, :infinity)
end
@impl true
def init(:ok) do
Process.flag(:trap_exit, true)
{:ok, {%{}, %{}}}
end
@impl true
def handle_call({:watch, supervisor, args}, {caller_pid, _ref}, {caller_refs, started_refs}) do
case DynamicSupervisor.start_child(supervisor, args) do
{:ok, started_pid} ->
Process.link(caller_pid)
caller_ref = Process.monitor(caller_pid)
started_ref = Process.monitor(started_pid)
caller_refs = Map.put(caller_refs, caller_ref, {supervisor, started_pid, started_ref})
started_refs = Map.put(started_refs, started_ref, {caller_pid, caller_ref})
{:reply, {:ok, started_pid}, {caller_refs, started_refs}}
other ->
{:reply, other, {caller_refs, started_refs}}
end
end
@impl true
def handle_info({:DOWN, ref, _, _, _}, {caller_refs, started_refs}) do
case caller_refs do
%{^ref => {supervisor, started_pid, started_ref}} ->
Process.demonitor(started_ref, [:flush])
DynamicSupervisor.terminate_child(supervisor, started_pid)
{:noreply, {Map.delete(caller_refs, ref), Map.delete(started_refs, started_ref)}}
%{} ->
%{^ref => {caller_pid, caller_ref}} = started_refs
Process.demonitor(caller_ref, [:flush])
Process.exit(caller_pid, :kill)
{:noreply, {Map.delete(caller_refs, caller_ref), Map.delete(started_refs, ref)}}
end
end
def handle_info({:EXIT, _, _}, state) do
{:noreply, state}
end
@impl true
def terminate(_, {_, started_refs}) do
for {_, {caller_pid, _}} <- started_refs do
Process.exit(caller_pid, :kill)
end
:ok
end
end

View File

@@ -0,0 +1,84 @@
defmodule DBConnection.Mixfile do
use Mix.Project
@source_url "https://github.com/elixir-ecto/db_connection"
@pools [:connection_pool, :ownership]
@version "2.9.0"
def project do
[
app: :db_connection,
version: @version,
elixir: "~> 1.11",
deps: deps(),
docs: docs(),
description: description(),
package: package(),
build_per_environment: false,
consolidate_protocols: false,
test_paths: test_paths(Mix.env()),
aliases: ["test.all": ["test", "test.pools"], "test.pools": &test_pools/1],
preferred_cli_env: ["test.all": :test]
]
end
def application do
[
extra_applications: [:logger],
mod: {DBConnection.App, []}
]
end
defp deps do
[
{:ex_doc, ">= 0.0.0", only: :dev, runtime: false},
{:telemetry, "~> 0.4 or ~> 1.0"}
]
end
defp docs do
[
source_url: @source_url,
source_ref: "v#{@version}",
main: DBConnection,
extras: ["CHANGELOG.md"]
]
end
defp description do
"""
Database connection behaviour for database transactions and connection pooling
"""
end
defp package do
%{
licenses: ["Apache-2.0"],
maintainers: ["James Fish", "José Valim"],
links: %{"GitHub" => @source_url}
}
end
defp test_paths(pool) when pool in @pools, do: ["integration_test/#{pool}"]
defp test_paths(_), do: ["test"]
defp test_pools(args) do
for env <- @pools, do: env_run(env, args)
end
defp env_run(env, args) do
args = if IO.ANSI.enabled?(), do: ["--color" | args], else: ["--no-color" | args]
IO.puts("==> Running tests for MIX_ENV=#{env} mix test")
{_, res} =
System.cmd("mix", ["test" | args],
into: IO.binstream(:stdio, :line),
env: [{"MIX_ENV", to_string(env)}]
)
if res > 0 do
System.at_exit(fn _ -> exit({:shutdown, 1}) end)
end
end
end